[PATCH] st: clear ILI if Medium Error

[Kai Makisara]

Some drives set the ILI flag together with MEDIUM ERROR
sense code. Clear the ILI flag in this case so that the
medium error will be handled. The problem was reported by
Maurizio Lombardi.

Signed-off-by: Kai Mäkisara 
---
 drivers/scsi/st.c |9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

--- a/drivers/scsi/st.c 2016-04-17 21:22:15.671897001 +0300
+++ b/drivers/scsi/st.c 2016-04-17 22:25:39.234321293 +0300
@@ -1974,9 +1974,12 @@ static long read_tape(struct scsi_tape *
transfer = (int)cmdstatp->uremainder64;
else
transfer = 0;
-   if (STp->block_size == 0 &&
-   cmdstatp->sense_hdr.sense_key == 
MEDIUM_ERROR)
-   transfer = bytes;
+   if (cmdstatp->sense_hdr.sense_key == 
MEDIUM_ERROR) {
+   if (STp->block_size == 0)
+   transfer = bytes;
+   /* Some drives set ILI with MEDIUM 
ERROR */
+   cmdstatp->flags &= ~SENSE_ILI;
+   }
 
if (cmdstatp->flags & SENSE_ILI) {  /* ILI 
*/
if (STp->block_size == 0 &&

[PATCH] st: Fix MTMKPART to work with newer drives

[Kai Makisara]

Change the MTMKPART operation of the MTIOCTOP ioctl so that it works also with
current drives (LTO-5/6, etc.). Send a separate FORMAT MEDIUM command if the
partition mode page indicates that this is required. Use LOAD to position the
tape at the beginning of tape.

The operation is extended so that if the argument is negative, its absolute
value specifies the size of partition 0, which is the physically first
partition of the tape.

Signed-off-by: Kai Mäkisara <kai.makis...@kolumbus.fi>
Tested-by: Shane M Seymour <shane.seym...@hpe.com>
Tested-by: Laurence Oberman <lober...@redhat.com>
Tested-by: Douglas Gilbert <dgilb...@interlog.com>
---
 Documentation/scsi/st.txt |   15 ++--
 drivers/scsi/st.c |  122 
 2 files changed, 121 insertions(+), 16 deletions(-)

--- a/drivers/scsi/st.c 2015-12-21 18:54:05.068882001 +0200
+++ b/drivers/scsi/st.c 2016-02-09 20:57:52.614816514 +0200
@@ -9,7 +9,7 @@
Steve Hirsch, Andreas Koppenh"ofer, Michael Leodolter, Eyal Lebedinsky,
Michael Schaefer, J"org Weule, and Eric Youngdale.
 
-   Copyright 1992 - 2010 Kai Makisara
+   Copyright 1992 - 2016 Kai Makisara
email kai.makis...@kolumbus.fi
 
Some small formal changes - aeb, 950809
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch <rgo...@atnf.csiro.au> Devfs 
support
  */
 
-static const char *verstr = "20101219";
+static const char *verstr = "20160209";
 
 #include 
 
@@ -3296,7 +3296,10 @@ static int switch_partition(struct scsi_
 #define PP_OFF_RESERVED7
 
 #define PP_BIT_IDP 0x20
+#define PP_BIT_FDP 0x80
 #define PP_MSK_PSUM_MB 0x10
+#define PP_MSK_PSUM_UNITS  0x18
+#define PP_MSK_POFM0x04
 
 /* Get the number of partitions on the tape. As a side effect reads the
mode page into the tape buffer. */
@@ -3322,6 +3325,29 @@ static int nbr_partitions(struct scsi_ta
 }
 
 
+static int format_medium(struct scsi_tape *STp, int format)
+{
+   int result = 0;
+   int timeout = STp->long_timeout;
+   unsigned char scmd[MAX_COMMAND_SIZE];
+   struct st_request *SRpnt;
+
+   memset(scmd, 0, MAX_COMMAND_SIZE);
+   scmd[0] = FORMAT_UNIT;
+   scmd[2] = format;
+   if (STp->immediate) {
+   scmd[1] |= 1;   /* Don't wait for completion */
+   timeout = STp->device->request_queue->rq_timeout;
+   }
+   DEBC_printk(STp, "Sending FORMAT MEDIUM\n");
+   SRpnt = st_do_scsi(NULL, STp, scmd, 0, DMA_NONE,
+  timeout, MAX_RETRIES, 1);
+   if (!SRpnt)
+   result = STp->buffer->syscall_result;
+   return result;
+}
+
+
 /* Partition the tape into two partitions if size > 0 or one partition if
size == 0.
 
@@ -3340,11 +3366,16 @@ static int nbr_partitions(struct scsi_ta
and 10 when 1 partition is defined (information from Eric Lee Green). This 
is
is acceptable also to some other old drives and enforced if the first 
partition
size field is used for the first additional partition size.
+
+   For drives that advertize SCSI-3 or newer, use the SSC-3 methods.
  */
 static int partition_tape(struct scsi_tape *STp, int size)
 {
int result;
+   int target_partition;
+   bool scsi3 = STp->device->scsi_level >= SCSI_3, needs_format = false;
int pgo, psd_cnt, psdo;
+   int psum = PP_MSK_PSUM_MB, units = 0;
unsigned char *bp;
 
result = read_mode_page(STp, PART_PAGE, 0);
@@ -3352,6 +3383,12 @@ static int partition_tape(struct scsi_ta
DEBC_printk(STp, "Can't read partition mode page.\n");
return result;
}
+   target_partition = 1;
+   if (size < 0) {
+   target_partition = 0;
+   size = -size;
+   }
+
/* The mode page is in the buffer. Let's modify it and write it. */
bp = (STp->buffer)->b_data;
pgo = MODE_HEADER_LENGTH + bp[MH_OFF_BDESCS_LENGTH];
@@ -3359,9 +3396,52 @@ static int partition_tape(struct scsi_ta
bp[pgo + MP_OFF_PAGE_LENGTH] + 2);
 
psd_cnt = (bp[pgo + MP_OFF_PAGE_LENGTH] + 2 - PART_PAGE_FIXED_LENGTH) / 
2;
+
+   if (scsi3) {
+   needs_format = (bp[pgo + PP_OFF_FLAGS] & PP_MSK_POFM) != 0;
+   if (needs_format && size == 0) {
+   /* No need to write the mode page when clearing
+*  partitioning
+*/
+   DEBC_printk(STp, "Formatting tape with one 
partition.\n");
+   result = format_medium(STp, 0);
+   goto out;
+   }
+   if (needs_format)  /* Leave the old value for HP DATs claiming 
SCSI_3 */
+   psd_cnt = 2;
+   if ((bp[pgo + PP_OFF_FLAGS] & PP_MSK_PSUM

RE: What partition should the MTMKPART argument specify? Was: Re: st driver doesn't seem to grok LTO partitioning

[Kai Makisara]

 8
>[ 3957.373732] st 8:0:0:0: [st2] Density 5a, tape length: 0, drv buffer: 1
>[ 3957.373734] st 8:0:0:0: [st2] Block size: 0, buffer size: 4096 (1 blocks).
>[ 3957.373750] st 8:0:0:0: [st2] Loading tape.
>[ 3957.388599] st 8:0:0:0: [st2] Block limits 1 - 16777215 bytes.
>[ 3957.389100] st 8:0:0:0: [st2] Mode sense. Length 11, medium 0, WBS 10, BLL 8
>[ 3957.389104] st 8:0:0:0: [st2] Density 5a, tape length: 0, drv buffer: 1
>[ 3957.389108] st 8:0:0:0: [st2] Block size: 0, buffer size: 4096 (1 blocks).
>[ 3957.390012] st 8:0:0:0: [st2] Partition page length is 16 bytes.
>[ 3957.390015] st 8:0:0:0: [st2] PP: max 3, add 0, xdp 1, psum 03, pofmetc 4, 
>rec 03, units 09, sizes: 2620 0
>[ 3957.390018] st 8:0:0:0: [st2] MP: 11 0e 03 00 3c 03 09 00 0a 3c 00 00
>[ 3957.390024] st 8:0:0:0: [st2] psd_cnt 2, max.parts 3, nbr_parts 0
>[ 3957.390027] st 8:0:0:0: [st2] Formatting tape with two partitions (1 = 2000 
>MB).
>[ 3957.390030] st 8:0:0:0: [st2] Sent partition page length is 12 bytes. 
>needs_format: 1
>[ 3957.390035] st 8:0:0:0: [st2] PP: max 3, add 1, xdp 1, psum 03, pofmetc 4, 
>rec 03, units 09, sizes: 2 0
>[ 3957.390039] st 8:0:0:0: [st2] MP: 11 0a 03 01 3c 03 09 00 00 02 00 00
>[ 3957.391184] st 8:0:0:0: [st2] Sending FORMAT MEDIUM
>[ 3957.398667] st 8:0:0:0: [st2] Error: 802, cmd: 4 0 1 0 0 0
>[ 3957.398672] st 8:0:0:0: [st2] Sense Key : Illegal Request [current]
>[ 3957.398676] st 8:0:0:0: [st2] Add. Sense: Parameter value invalid
>[ 3957.398684] st 8:0:0:0: [st2] Rewinding tape.
>
>Again I would have expected 0x and 2 not 2 and 0.
>
>If I ask for -2000 then it works:
>
...
>
>This code appears to be wrong:
>
>bp[psdo] = (size >> 8) & 0xff;
>bp[psdo + 1] = size & 0xff;
>if (target_partition == 0)
>bp[psdo + 2] = bp[psdo + 3] = 0xff;
>
The problem is not here. This code just adds the contents to the second 
psd. The problem is in this condition and only if maximum number of 
partitions exceeds one (as it does for LTO-6):

if (target_partition > 0 && psd_cnt > bp[pgo + PP_OFF_MAX_ADD_PARTS]) {
bp[psdo] = bp[psdo + 1] = 0xff;  /* Rest of the tape to 
partition 0 */
psdo += 2;
}

At the end of this message is a new patch where I have modified this 
condition slightly. The new version seems to make correct mode pages 
when I tested it by inserting code that modified the mode page returned 
by my drive to mimic other drives.

...
>For posterity in case anyone reads this at a later point just to 
>explain the size values for DDS - while the DDS sizes may look wrong 
>they are not. The second size value is not sent to the tape drive only 
>the first size is since most (all?) DDS drives only support setting an 
>explicit size for the second partition only and partition 0 
>automatically gets the rest of the tape.
>
The Sony DDS-2 drive did use two psds. In addition to the DDS drives, 
partitioning did work for several other drive types in the last century.

>If the changes are correct and needed and you want to add those two 
>changes please do so. I've been looking very carefully at the code 
>changes as part of the testing effort and happy to add any combination 
>of reviewed-by and tested-by to the patch. If required feel free to 
>change it as needed. I'll retest a consolidated version of the patch if 
>you add the changes and send it out.

I added the two debug printk changes you suggested.

>
>Laurence, when you get your LTO drive test without my changes in to 
>make sure you see the same issue and then if you do test with the 
>changes I added above. That's just to double check that there's nothing 
>wrong with my setup and the code changes are required.

Thanks,
Kai

---8<
--- ref/drivers/scsi/st.c   2015-12-21 18:54:05.068882001 +0200
+++ new/drivers/scsi/st.c   2016-02-03 20:31:33.709018742 +0200
@@ -9,7 +9,7 @@
Steve Hirsch, Andreas Koppenh"ofer, Michael Leodolter, Eyal Lebedinsky,
Michael Schaefer, J"org Weule, and Eric Youngdale.
 
-   Copyright 1992 - 2010 Kai Makisara
+   Copyright 1992 - 2016 Kai Makisara
email kai.makis...@kolumbus.fi
 
Some small formal changes - aeb, 950809
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch <rgo...@atnf.csiro.au> Devfs 
support
  */
 
-static const char *verstr = "20101219";
+static const char *verstr = "20160203";
 
 #include 
 
@@ -3296,7 +3296,10 @@
 #define PP_OFF_RESERVED7
 
 #define PP_BIT_IDP 0x20
+#define PP_BIT_FDP 0x80
 #define PP_MSK_PSUM_MB 0x10
+#define PP_MSK_PSUM_UNITS  0x18
+#define PP_MSK_POFM0x04
 
 /* Get the number of partitions 

RE: What partition should the MTMKPART argument specify? Was: Re: st driver doesn't seem to grok LTO partitioning

[Kai Makisara]

On Friday 2016-01-22 04:10, Seymour, Shane M wrote:


>> -Original Message-
>> From: linux-scsi-ow...@vger.kernel.org [mailto:linux-scsi-
>> ow...@vger.kernel.org] On Behalf Of "Kai Mäkisara (Kolumbus)"
>> Sent: Friday, January 22, 2016 7:59 AM
>> To: Seymour, Shane M
>> Cc: Laurence Oberman; Emmanuel Florac; Laurence Oberman; linux-
>> s...@vger.kernel.org
>> Subject: What partition should the MTMKPART argument specify? Was: Re:
>> st driver doesn't seem to grok LTO partitioning
>> 
...
>> 
>> There seem to be lot of arguments supporting both possible choices. Should
>> we use the existing definition (1) or change it for the drives supporting 
>> SCSI
>> level >= 3 (or supporting FORMAT MEDIUM)? The definition can’t be
>> changed later. This is why we should make a good decision.
>> 
>> Opinions?
>
>How about using the fact the size is signed to indicate one slightly different
>thing? I'm not sure if you'd call this using or abusing the fact that it's 
>signed.
>
>Make the default behavior for a positive size the same as the current
>behavior for SCSI-2 (size applies to partition 1). If the size is negative then
>the absolute value of the size applies to partition 0. That provides some
>flexibility in choosing which partition the size applies to if it worked that
>way for all devices.
>
>With that you also get consistent behavior between tape drives without
>having to know if the size will apply to partition 0 or partition 1 based on
>the tape technology and you get the benefit of being able to set an
>explicit size for partition 0 or partition 1.
>
I like this suggestion, because of the reasons you point out. I think 
this is using the fact that the argument is signed.

>You could overload the value of 0 as well to use FDP to choose the sizes
>for the partitions (assuming 0 doesn't already have a side effect in
>the code). Then you get whatever the tape drive wants to do.
>
The value zero is used to specify only one partition. The previous 
patches overloaded the size 1. However, I think supporting FDP is 
useless nowadays: the drives that support FDP=1 seem to end up with the 
same partitioning (two wraps to partition 1) with any small number as 
argument.

Below is a test patch that implements the current behaviour with 
non-negative argument (but works with LTOs etc.) and makes partition 
zero size absolute value of argument (MB) if argument is negative. If 
you want to test the patch, note that the current mt-st does not accept 
negative numbers. A minimal patch is needed.

Thanks,
Kai
---8<
--- ref/drivers/scsi/st.c   2015-12-21 18:54:05.068882001 +0200
+++ new/drivers/scsi/st.c   2016-01-24 22:36:13.250928500 +0200
@@ -9,7 +9,7 @@
Steve Hirsch, Andreas Koppenh"ofer, Michael Leodolter, Eyal Lebedinsky,
Michael Schaefer, J"org Weule, and Eric Youngdale.
 
-   Copyright 1992 - 2010 Kai Makisara
+   Copyright 1992 - 2016 Kai Makisara
email kai.makis...@kolumbus.fi
 
Some small formal changes - aeb, 950809
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch <rgo...@atnf.csiro.au> Devfs 
support
  */
 
-static const char *verstr = "20101219";
+static const char *verstr = "20160124";
 
 #include 
 
@@ -3296,7 +3296,10 @@
 #define PP_OFF_RESERVED7
 
 #define PP_BIT_IDP 0x20
+#define PP_BIT_FDP 0x80
 #define PP_MSK_PSUM_MB 0x10
+#define PP_MSK_PSUM_UNITS  0x18
+#define PP_MSK_POFM0x04
 
 /* Get the number of partitions on the tape. As a side effect reads the
mode page into the tape buffer. */
@@ -3322,6 +3325,29 @@
 }
 
 
+static int format_medium(struct scsi_tape *STp, int format)
+{
+   int result = 0;
+   int timeout = STp->long_timeout;
+   unsigned char scmd[MAX_COMMAND_SIZE];
+   struct st_request *SRpnt;
+
+   memset(scmd, 0, MAX_COMMAND_SIZE);
+   scmd[0] = FORMAT_UNIT;
+   scmd[2] = format;
+   if (STp->immediate) {
+   scmd[1] |= 1;   /* Don't wait for completion */
+   timeout = STp->device->request_queue->rq_timeout;
+   }
+   DEBC_printk(STp, "Sending FORMAT MEDIUM\n");
+   SRpnt = st_do_scsi(NULL, STp, scmd, 0, DMA_NONE,
+  timeout, MAX_RETRIES, 1);
+   if (!SRpnt)
+   result = STp->buffer->syscall_result;
+   return result;
+}
+
+
 /* Partition the tape into two partitions if size > 0 or one partition if
size == 0.
 
@@ -3340,11 +3366,16 @@
and 10 when 1 partition is defined (information from Eric Lee Green). This 
is
is acceptable also to some other old drives and enforced if the first 
partition
size field is used for the first additio

Re: st driver doesn't seem to grok LTO partitioning

[Kai Makisara]

On Thursday 2015-12-31 18:08, "Kai Mäkisara (Kolumbus)" wrote:

...
>In the HP LTFS sources I found an interesting detail: the code does LOAD 
>before unformatting.
>A comment says that it is in some cases better method to put the position to 
>beginning of
>partition 0 than other methods. You could try ‘mt load’ before trying to 
>partition.
>
Here is again a new version of the patch. This does load before 
partitioning. The code performing default partitioning (FDP=1) has also 
been slightly modified (two more bits of the original mode page 
retained).

The patch has been tested with my DDS-4 drive.

Kai
-8<
--- ref/drivers/scsi/st.c   2015-12-21 18:54:05.068882001 +0200
+++ new/drivers/scsi/st.c   2016-01-04 12:09:07.702559815 +0200
@@ -9,7 +9,7 @@
Steve Hirsch, Andreas Koppenh"ofer, Michael Leodolter, Eyal Lebedinsky,
Michael Schaefer, J"org Weule, and Eric Youngdale.
 
-   Copyright 1992 - 2010 Kai Makisara
+   Copyright 1992 - 2016 Kai Makisara
email kai.makis...@kolumbus.fi
 
Some small formal changes - aeb, 950809
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch <rgo...@atnf.csiro.au> Devfs 
support
  */
 
-static const char *verstr = "20101219";
+static const char *verstr = "20160104";
 
 #include 
 
@@ -3296,7 +3296,10 @@
 #define PP_OFF_RESERVED7
 
 #define PP_BIT_IDP 0x20
+#define PP_BIT_FDP 0x80
 #define PP_MSK_PSUM_MB 0x10
+#define PP_MSK_PSUM_UNITS  0x18
+#define PP_MSK_POFM0x04
 
 /* Get the number of partitions on the tape. As a side effect reads the
mode page into the tape buffer. */
@@ -3322,6 +3325,29 @@
 }
 
 
+static int format_medium(struct scsi_tape *STp, int format)
+{
+   int result = 0;
+   int timeout = STp->long_timeout;
+   unsigned char scmd[MAX_COMMAND_SIZE];
+   struct st_request *SRpnt;
+
+   memset(scmd, 0, MAX_COMMAND_SIZE);
+   scmd[0] = FORMAT_UNIT;
+   scmd[2] = format;
+   if (STp->immediate) {
+   scmd[1] |= 1;   /* Don't wait for completion */
+   timeout = STp->device->request_queue->rq_timeout;
+   }
+   DEBC_printk(STp, "Sending FORMAT MEDIUM\n");
+   SRpnt = st_do_scsi(NULL, STp, scmd, 0, DMA_NONE,
+  timeout, MAX_RETRIES, 1);
+   if (!SRpnt)
+   result = STp->buffer->syscall_result;
+   return result;
+}
+
+
 /* Partition the tape into two partitions if size > 0 or one partition if
size == 0.
 
@@ -3340,11 +3366,15 @@
and 10 when 1 partition is defined (information from Eric Lee Green). This 
is
is acceptable also to some other old drives and enforced if the first 
partition
size field is used for the first additional partition size.
+
+   For drives that advertize SCSI-3 or newer, use the SSC-3 methods.
  */
 static int partition_tape(struct scsi_tape *STp, int size)
 {
int result;
+   bool scsi3 = STp->device->scsi_level >= SCSI_3,  needs_format = false;
int pgo, psd_cnt, psdo;
+   int psum = PP_MSK_PSUM_MB, units = 0;
unsigned char *bp;
 
result = read_mode_page(STp, PART_PAGE, 0);
@@ -3357,11 +3387,58 @@
pgo = MODE_HEADER_LENGTH + bp[MH_OFF_BDESCS_LENGTH];
DEBC_printk(STp, "Partition page length is %d bytes.\n",
bp[pgo + MP_OFF_PAGE_LENGTH] + 2);
+   DEBC_printk(STp, "PP: max %u, add %u, xdp %u, psum %02x, pofmetc %u, "
+   "rec %02x, units %02x, sizes: %u %u\n",
+   bp[pgo + PP_OFF_MAX_ADD_PARTS],
+   bp[pgo + PP_OFF_NBR_ADD_PARTS],
+   (bp[pgo + PP_OFF_FLAGS] & 0xe0) >> 5,
+   (bp[pgo + PP_OFF_FLAGS] & 0x18) >> 3,
+   bp[pgo + PP_OFF_FLAGS] & 0x07,
+   bp[pgo + 5],
+   bp[pgo + PP_OFF_PART_UNITS],
+   bp[pgo + 8] * 256 + bp[pgo + 9],
+   bp[pgo + 10] * 256 + bp[pgo + 11]);
+   DEBC_printk(STp, "MP: %02x %02x %02x %02x %02x %02x %02x %02x %02x %02x 
%02x %02x\n",
+   bp[pgo], bp[pgo+1], bp[pgo+2], bp[pgo+3], bp[pgo+4], 
bp[pgo+5],
+   bp[pgo+6], bp[pgo+7], bp[pgo+8], bp[pgo+9], bp[pgo+10], 
bp[pgo+11]);
+
+   if (scsi3) {
+   needs_format = (bp[pgo + PP_OFF_FLAGS] & PP_MSK_POFM) != 0;
+   if (needs_format && size == 0) {
+   /* No need to write the mode page in this case */
+   result = format_medium(STp, 0);
+   goto out;
+   }
+
+   psd_cnt = size > 0 ? 2 : 1;
+   if ((bp[pgo + PP_OFF_FLAGS] & PP_MSK_PSUM_UNITS) == 
PP_MSK_PSUM_UNITS) {
+   /* Use

Re: st driver doesn't seem to grok LTO partitioning

[Kai Makisara]

On Tuesday 2015-12-29 20:13, Emmanuel Florac wrote:

...
>It works when providing the size :
>
># mt -f /dev/nst0  mkpartition 36764
>
Good.

I think I found out why it did not work with the default format. At the 
end of this message you find a new patch that should correct that. There 
are also other changes:
- some changes when specifying the size in the mode page; needed for the
  IBM 3592 drives but should not change results with LTO
- formatting with one partition is done directly with FORMAT MEDIUM if
  the drive requires that command
- some other bugs fixed

Thanks for testing,
Kai

---8<
--- ref/drivers/scsi/st.c   2015-12-21 18:54:05.068882001 +0200
+++ new/drivers/scsi/st.c   2015-12-30 19:30:37.900152216 +0200
@@ -9,7 +9,7 @@
Steve Hirsch, Andreas Koppenh"ofer, Michael Leodolter, Eyal Lebedinsky,
Michael Schaefer, J"org Weule, and Eric Youngdale.
 
-   Copyright 1992 - 2010 Kai Makisara
+   Copyright 1992 - 2016 Kai Makisara
email kai.makis...@kolumbus.fi
 
Some small formal changes - aeb, 950809
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch <rgo...@atnf.csiro.au> Devfs 
support
  */
 
-static const char *verstr = "20101219";
+static const char *verstr = "20151230";
 
 #include 
 
@@ -3296,7 +3296,10 @@
 #define PP_OFF_RESERVED7
 
 #define PP_BIT_IDP 0x20
+#define PP_BIT_FDP 0x80
 #define PP_MSK_PSUM_MB 0x10
+#define PP_MSK_PSUM_UNITS  0x18
+#define PP_MSK_POFM0x04
 
 /* Get the number of partitions on the tape. As a side effect reads the
mode page into the tape buffer. */
@@ -3322,6 +3325,29 @@
 }
 
 
+static int format_medium(struct scsi_tape *STp, int format)
+{
+   int result = 0;
+   int timeout = STp->long_timeout;
+   unsigned char scmd[MAX_COMMAND_SIZE];
+   struct st_request *SRpnt;
+
+   memset(scmd, 0, MAX_COMMAND_SIZE);
+   scmd[0] = FORMAT_UNIT;
+   scmd[2] = format;
+   if (STp->immediate) {
+   scmd[1] |= 1;   /* Don't wait for completion */
+   timeout = STp->device->request_queue->rq_timeout;
+   }
+   DEBC_printk(STp, "Sending FORMAT MEDIUM\n");
+   SRpnt = st_do_scsi(NULL, STp, scmd, 0, DMA_NONE,
+  timeout, MAX_RETRIES, 1);
+   if (!SRpnt)
+   result = STp->buffer->syscall_result;
+   return result;
+}
+
+
 /* Partition the tape into two partitions if size > 0 or one partition if
size == 0.
 
@@ -3340,11 +3366,15 @@
and 10 when 1 partition is defined (information from Eric Lee Green). This 
is
is acceptable also to some other old drives and enforced if the first 
partition
size field is used for the first additional partition size.
+
+   For drives that advertize SCSI-3 or newer, use the SSC-3 methods.
  */
 static int partition_tape(struct scsi_tape *STp, int size)
 {
int result;
+   bool scsi3 = STp->device->scsi_level >= SCSI_3,  needs_format = false;
int pgo, psd_cnt, psdo;
+   int psum = PP_MSK_PSUM_MB, units = 0;
unsigned char *bp;
 
result = read_mode_page(STp, PART_PAGE, 0);
@@ -3357,11 +3387,54 @@
pgo = MODE_HEADER_LENGTH + bp[MH_OFF_BDESCS_LENGTH];
DEBC_printk(STp, "Partition page length is %d bytes.\n",
bp[pgo + MP_OFF_PAGE_LENGTH] + 2);
+   DEBC_printk(STp, "PP: max %u, add %u, xdp %u, psum %02x, pofmetc %u, "
+   "rec %02x, units %02x, sizes: %u %u\n",
+   bp[pgo + PP_OFF_MAX_ADD_PARTS],
+   bp[pgo + PP_OFF_NBR_ADD_PARTS],
+   (bp[pgo + PP_OFF_FLAGS] & 0xe0) >> 5,
+   (bp[pgo + PP_OFF_FLAGS] & 0x18) >> 3,
+   bp[pgo + PP_OFF_FLAGS] & 0x07,
+   bp[pgo + 5],
+   bp[pgo + PP_OFF_PART_UNITS],
+   bp[pgo + 8] * 256 + bp[pgo + 9],
+   bp[pgo + 10] * 256 + bp[pgo + 11]);
+
+   if (scsi3) {
+   needs_format = (bp[pgo + PP_OFF_FLAGS] & PP_MSK_POFM) != 0;
+   if (needs_format && size == 0) {
+   result = format_medium(STp, 0);
+   goto out;
+   }
+
+   psd_cnt = size > 0 ? 2 : 1;
+   if ((bp[pgo + PP_OFF_FLAGS] & PP_MSK_PSUM_UNITS) == 
PP_MSK_PSUM_UNITS) {
+   /* Use units scaling for large partitions if the device 
suggests
+  it and no precision lost. Required for IBM TS1140/50 
drives
+  that don't support MB units. */
+   if (size >= 1000 && (size % 1000) == 0) {
+   size /= 1000;
+   psum =

Re: st driver doesn't seem to grok LTO partitioning

[Kai Makisara]

On Monday 2015-12-21 20:57, "Kai Mäkisara (Kolumbus)" wrote:

...
>I can look at the manuals during Christmas holidays and try to think about a 
>suggestion.
>All other suggestions are, of course, welcome. My view may be somewhat 
>theoretical
>because I don’t have access to current hardware.
>
I have had some time to look at manuals and even do some coding :-)

The partitioning is initiated with the MTMKPART operation of the 
MTIOCTOP ioctl. This has one int parameter mt_count that is used to 
specify the size in megabytes of the first partition on the tape. Note 
that, for historical reasons, the number of this partition used to be 1, 
not zero. So, the ioctls allocates the specified space to partition 1 
and the rest to partition 0. This intefarce does not allow specifying 
more than two partitions. We can make work for current drives quite 
easily. More than two partitions need a new API but do we have a real 
need for this?

The current code did not work basically because the current drives need 
a separate FORMAT MEDIUM command to do the partitioning specified by 
MODE SELECT. There are smaller details that could have also been a 
problem.

The patch at the end of this message is an attempt to make the 
partitioning work for both old and new drives. The patch is against st.c 
from the current git kernel, although I have tested it in 4.3.3. It 
seems to work with my HP DDS-4 drive. It includes some debugging printks 
and I have tried to see that the commands that would have been sent to 
newer drives are basically correct.

The patch uses the scsi level of the device to separate processing. The 
FORMAT MEDIUM command is defined in SCSI-3 and I suppose that no current 
drive is still SCSI-2. In addition to the "sane" changes using the 
method to specify partitions, the patch implements the following: if the 
size is 1, the driver tells the drive to use default partitioning for 
two partitions. For the HP Ultrium this should result in partition 0 of 
1425 GB and 1 of 37.5 GB. I don't know if this is a useful addition.


Kai

8<--
--- ref/drivers/scsi/st.c   2015-12-21 18:54:05.068882001 +0200
+++ new/drivers/scsi/st.c   2015-12-25 16:49:54.345263675 +0200
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch  Devfs 
support
  */
 
-static const char *verstr = "20101219";
+static const char *verstr = "20151225";
 
 #include 
 
@@ -3296,7 +3296,10 @@
 #define PP_OFF_RESERVED7
 
 #define PP_BIT_IDP 0x20
+#define PP_BIT_FDP 0x80
 #define PP_MSK_PSUM_MB 0x10
+#define PP_MSK_PSUM_UNITS  0x18
+#define PP_MSK_POFM0x04
 
 /* Get the number of partitions on the tape. As a side effect reads the
mode page into the tape buffer. */
@@ -3344,7 +3347,9 @@
 static int partition_tape(struct scsi_tape *STp, int size)
 {
int result;
+   int scsi3 = STp->device->scsi_level >= SCSI_3, needs_format = 0;
int pgo, psd_cnt, psdo;
+   int psum = PP_MSK_PSUM_MB, units = 0;
unsigned char *bp;
 
result = read_mode_page(STp, PART_PAGE, 0);
@@ -3357,10 +3362,32 @@
pgo = MODE_HEADER_LENGTH + bp[MH_OFF_BDESCS_LENGTH];
DEBC_printk(STp, "Partition page length is %d bytes.\n",
bp[pgo + MP_OFF_PAGE_LENGTH] + 2);
-
-   psd_cnt = (bp[pgo + MP_OFF_PAGE_LENGTH] + 2 - PART_PAGE_FIXED_LENGTH) / 
2;
+   DEBC_printk(STp, "PP: max %u, add %u, xdp %u, psum %02x, pofmetc %u,"
+   "rec %02x units %02x, sizes: %u %u\n",
+   bp[pgo + PP_OFF_MAX_ADD_PARTS],
+   bp[pgo + PP_OFF_NBR_ADD_PARTS],
+   (bp[pgo + PP_OFF_FLAGS] & 0xe0) >> 5,
+   (bp[pgo + PP_OFF_FLAGS] & 0x18) >> 3,
+   bp[pgo + PP_OFF_FLAGS] & 0x03,
+   bp[pgo + 5],
+   bp[pgo + PP_OFF_PART_UNITS],
+   bp[pgo + 8] * 256 + bp[pgo + 9],
+   bp[pgo + 10] * 256 + bp[pgo + 11]);
+
+   if (scsi3) {
+   psd_cnt = size > 0 ? 2 : 1;
+   if (size >= 65535) {
+   size /= 1000;
+   psum = PP_MSK_PSUM_UNITS;
+   units = 9; /* GB */
+   }
+   } else
+   psd_cnt = (bp[pgo + MP_OFF_PAGE_LENGTH] + 2 -
+  PART_PAGE_FIXED_LENGTH) / 2;
+   if (size >= 65535)
+   return -EINVAL;
psdo = pgo + PART_PAGE_FIXED_LENGTH;
-   if (psd_cnt > bp[pgo + PP_OFF_MAX_ADD_PARTS]) {
+   if (scsi3 || psd_cnt > bp[pgo + PP_OFF_MAX_ADD_PARTS]) {
bp[psdo] = bp[psdo + 1] = 0xff;  /* Rest of the tape */
psdo += 2;
}
@@ -3375,6 +3402,11 @@
if (psd_cnt <= bp[pgo + PP_OFF_MAX_ADD_PARTS])
bp[pgo + MP_OFF_PAGE_LENGTH] = 6;
DEBC_printk(STp, "Formatting tape with one partition.\n");

Re: Open/INQUIRY fails on RESERVE'd tape device

[kai . makisara]

Matthias Eble [psychotr...@gmail.com] kirjoitti: 

Hi list,

When a tape device is reserved with old reserve/release commands,
we see inquiry only works on the scsi generic device. For scsi tape devices
open() fails already:

# lsscsi -g | grep st15
[2:0:6:0]tapeHP   Ultrium 5-SCSI   I5DZ  /dev/st15  /dev/sg17

# sg_vpd -vvv /dev/st15
  open /dev/st15 with flags=0x800
  error opening file: /dev/st15: Input/output error

# sg_vpd -vvv /dev/nst15
  open /dev/nst15 with flags=0x800
  error opening file: /dev/nst15: Input/output error

# sg_vpd -vvv /dev/sg17
open /dev/sg17 with flags=0x800
Supported VPD pages VPD page:
inquiry cdb: 12 01 00 00 fc 00
  duration=2 ms
inquiry: requested 252 bytes but got 22 bytes
   [PQual=0  Peripheral device type: tape]
  Supported VPD pages [sv]
  Unit serial number [sn]
  ...


So: should open() fail on a reserved tape device?
SPC2 states that INQUIRY should never conflict.
Or does that only apply to the generic device?
Okay, it doesn't conflict, but open fails. A SunOS st man page I found
states, INQUIRY shall be possible with reserved devices.


Opening a st device does more than INQUIRY (TEST_UNIT_READY, READ_BLOCK_LIMITS,
MODE_SENSE). Can this explain what you see?

Kai


--



--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: PATCH: st.c: Fix blk_get_queue usage

[Kai Makisara]

On Thu, 14 Nov 2013, Bodo Stroesser wrote:

...
 From: Bodo Stroesser bstroes...@ts.fujitsu.com
 Date: Thu, 14 Nov 2013 14:35:00 +0100
 Subject: [PATCH] sg: fix blk_get_queue usage
 
 If blk_queue_get() in st_probe fails, disk-queue must not
 be set to SDp-request_queue, as that would result in
 put_disk() dropping a not taken reference.
 
 Thus, disk-queue should be set only after a successful
 blk_queue_get().
 Revised patch due to a hint from Kai Makisara.
 
 Signed-off-by: Bodo Stroesser bstroes...@ts.fujitsu.com

Acked-by: Kai Mäkisara kai.makis...@kolumbus.fi

 
 ---
 
 --- a/drivers/scsi/st.c   2013-11-14 14:10:40.0 +0100
 +++ b/drivers/scsi/st.c   2013-11-14 14:10:57.0 +0100
 @@ -4107,11 +4107,11 @@
   kref_init(tpnt-kref);
   tpnt-disk = disk;
   disk-private_data = tpnt-driver;
 - disk-queue = SDp-request_queue;
   /* SCSI tape doesn't register this gendisk via add_disk().  Manually
* take queue reference that release_disk() expects. */
 - if (blk_get_queue(disk-queue))
 + if (blk_get_queue(SDp-request_queue))
   goto out_put_disk;
 + disk-queue = SDp-request_queue;
   tpnt-driver = st_template;
  
   tpnt-device = SDp;

Thanks for the fix,
Kai

Re: request size limit in scsi tape driver

[Kai Makisara]

On Mon, 20 Aug 2012, Olaf Hering wrote:

 
 Currently the st driver does not to break up write requests into smaller
 chunks to satisfy the -max_hw_sectors limit of the underlying host driver.
 
  # modprobe -v scsi_debug ptype=1 opts=1 dev_size_mb=123
  # dd if=/dev/zero of=/dev/st0 bs=$((1024*54321))
 
 This leads to -EBUSY from this call chain:
 st_write - st_do_scsi - st_scsi_execute - blk_rq_map_user
 
 What is the reason for this behaviour?
 Should st_write write in smaller chunks, or would that break real
 hardware because they expect certain block sizes?
 
In variable block mode the write() byte count determines the size of the 
tape block. Exactly one SCSI command is used to write one block. This is 
why the writes can't be split in variable block mode. In fixed block mode 
the writes are split but this results in several tape blocks for each 
write().

Kai
--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [patch 0/5] st: Clean up and raise max device limit (v4)

[Kai Makisara]

On Sat, 18 Aug 2012, Jeff Mahoney wrote:

 This patchset cleans up the SCSI tape device handling code and leverages it
 to lift the limitation of the number of tape drives from the previous
 arbitrary limit of 128 to the maximum supported by a device node that
 creates 8 character devices per physical device. Since minors are 20 bits,
 that means 2^17 tape drives can be supported.
 
 Changed in this version: A previous revision introduced a regression where
 tape drives would not be shown as tape devices in lsscsi. This was due to
 the missing tape symlink in sysfs. That issue has been addressed and
 lsscsi works properly again. Also, it passes checkpatch with no errors.
 
 Please apply.
 
The whole series (1-5):

Acked-by: Kai Mäkisara kai.makis...@kolumbus.fi

Thanks,
Kai

Re: [PATCH] st: unblock concurrent access in st_open()

[Kai Makisara]

On Wed, 15 Aug 2012, Hannes Reinecke wrote:

 st_open() is taking a global mutex 'st_mutex' to protect
 against concurrent calls to st_open(). However, as
 st_open() might take quite some time before returning,
 _any_ open to _any_ st device will be blocked during
 that time.
 
 Seeing that all critical sections are already protected
 by individual spinlocks 'st_mutex' doesn't serve any
 purpose and can be removed, unblocking concurrent
 accesses.
 
 Signed-off-by: Hannes Reinecke h...@suse.de
 Cc: James Bottomley jbottom...@parallels.com
 Cc: Kai Mäkisara kai.makis...@kolumbus.fi
 Cc: Lee Duncan ldun...@suse.com
 ---
  drivers/scsi/st.c |8 ++--
  1 files changed, 2 insertions(+), 6 deletions(-)
 
 diff --git a/drivers/scsi/st.c b/drivers/scsi/st.c
 index e41998c..c6c172e 100644
 --- a/drivers/scsi/st.c
 +++ b/drivers/scsi/st.c
...
 @@ -1262,16 +1258,16 @@ static int st_open(struct inode *inode, struct file 
 *filp)
   retval = (-EIO);
   goto err_out;
   }
 - mutex_unlock(st_mutex);
   return 0;
  
   err_out:
   normalize_buffer(STp-buffer);
 + write_lock(st_dev_arr_lock);
   STp-in_use = 0;
 + write_unlock(st_dev_arr_lock);

I am not sure locking here is necessary. No-one should be able to remove 
the tape struct when in_use is non-zero. But I don't object to this extra 
protection.

Acked-by: Kai Mäkisara kai.makis...@kolumbus.fi

Thanks,
Kai

[PATCH 1/2] SCSI tape: add option to use SILI in variable block reads

[Kai Makisara]

Add new option MT_ST_SILI to enable setting the SILI bit in reads in variable
block mode. If SILI is set, reading a block shorter than the byte count does
not result in CHECK CONDITION. The length of the block is determined using the
residual count from the HBA. Avoiding the REQUEST SENSE command for every
block speeds up some real applications considerably.

Signed-off-by: Kai Makisara [EMAIL PROTECTED]
---
The patch is against Feb 24 git version of 2.5.25-rc2. Candidate for
inclusion into 2.6.26.

 Documentation/scsi/st.txt |7 ++-
 drivers/scsi/st.c |   40 
 drivers/scsi/st.h |3 +++
 drivers/scsi/st_options.h |6 +-
 include/linux/mtio.h  |1 +
 5 files changed, 51 insertions(+), 6 deletions(-)

Index: linux-2.6.25-rc2-q/drivers/scsi/st.c
===
--- linux-2.6.25-rc2-q.orig/drivers/scsi/st.c
+++ linux-2.6.25-rc2-q/drivers/scsi/st.c
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
  */
 
-static const char *verstr = 20080221;
+static const char *verstr = 20080224;
 
 #include linux/module.h
 
@@ -183,6 +183,7 @@ static int modes_defined;
 
 static struct st_buffer *new_tape_buffer(int, int, int);
 static int enlarge_buffer(struct st_buffer *, int, int);
+static void clear_buffer(struct st_buffer *);
 static void normalize_buffer(struct st_buffer *);
 static int append_to_buffer(const char __user *, struct st_buffer *, int);
 static int from_buffer(struct st_buffer *, char __user *, int);
@@ -442,6 +443,7 @@ static void st_sleep_done(void *data, ch
 
memcpy(SRpnt-sense, sense, SCSI_SENSE_BUFFERSIZE);
(STp-buffer)-cmdstat.midlevel_result = SRpnt-result = result;
+   (STp-buffer)-cmdstat.residual = resid;
DEB( STp-write_pending = 0; )
 
if (SRpnt-waiting)
@@ -1159,6 +1161,7 @@ static int st_open(struct inode *inode, 
goto err_out;
}
 
+   (STp-buffer)-cleared = 0;
(STp-buffer)-writing = 0;
(STp-buffer)-syscall_result = 0;
 
@@ -1432,8 +1435,14 @@ static int setup_buffering(struct scsi_t
if (STp-block_size)
bufsize = STp-block_size  st_fixed_buffer_size ?
STp-block_size : st_fixed_buffer_size;
-   else
+   else {
bufsize = count;
+   /* Make sure that data from previous user is not leaked 
even if
+  HBA does not return correct residual */
+   if (is_read  STp-sili  !STbp-cleared)
+   clear_buffer(STbp);
+   }
+
if (bufsize  STbp-buffer_size 
!enlarge_buffer(STbp, bufsize, STp-restr_dma)) {
printk(KERN_WARNING %s: Can't allocate %d byte tape 
buffer.\n,
@@ -1783,6 +1792,8 @@ static long read_tape(struct scsi_tape *
memset(cmd, 0, MAX_COMMAND_SIZE);
cmd[0] = READ_6;
cmd[1] = (STp-block_size != 0);
+   if (!cmd[1]  STp-sili)
+   cmd[1] |= 2;
cmd[2] = blks  16;
cmd[3] = blks  8;
cmd[4] = blks;
@@ -1911,8 +1922,11 @@ static long read_tape(struct scsi_tape *
 
}
/* End of error handling */ 
-   else/* Read successful */
+   else {  /* Read successful */
STbp-buffer_bytes = bytes;
+   if (STp-sili) /* In fixed block mode residual is always zero 
here */
+   STbp-buffer_bytes -= STp-buffer-cmdstat.residual;
+   }
 
if (STps-drv_block = 0) {
if (STp-block_size == 0)
@@ -2090,7 +2104,8 @@ static void st_log_options(struct scsi_t
   name, STm-defaults_for_writes, STp-omit_blklims, 
STp-can_partitions,
   STp-scsi2_logical);
printk(KERN_INFO
-  %s:sysv: %d nowait: %d\n, name, STm-sysv, 
STp-immediate);
+  %s:sysv: %d nowait: %d sili: %d\n, name, 
STm-sysv, STp-immediate,
+   STp-sili);
printk(KERN_INFO %s:debugging: %d\n,
   name, debugging);
}
@@ -2133,6 +2148,7 @@ static int st_set_options(struct scsi_ta
STp-scsi2_logical = (options  MT_ST_SCSI2LOGICAL) != 0;
STp-immediate = (options  MT_ST_NOWAIT) != 0;
STm-sysv = (options  MT_ST_SYSV) != 0;
+   STp-sili = (options  MT_ST_SILI) != 0;
DEB( debugging = (options  MT_ST_DEBUGGING) != 0;
 st_log_options(STp, STm, name); )
} else if (code == MT_ST_SETBOOLEANS || code == MT_ST_CLEARBOOLEANS) {
@@ -2164,6 +2180,8 @@ static int st_set_options(struct scsi_ta
STp-immediate = value;
if ((options  MT_ST_SYSV) != 0

[PATCH 2/2] SCSI tape: show options currently set in sysfs

[Kai Makisara]

Show the current binary tape driver and mode options is sysfs. A file
(options) is created in each directory in /sys/class/scsi_tape. The files
contain masks showing the options. The mask bit definitions are the same as
used when setting the options using the MTSETDRVBUFFER function in the
MTIOCTOP ioctl (defined in include/linux/mtio.h). For example:
 cat /sys/class/scsi_tape/nst0/options 
0x0d07

Signed-off-by: Kai Makisara [EMAIL PROTECTED]
---
The patch is against Feb 24 git version of 2.6.25-rc2. Candidate for
inclusion into 2.6.26.

 Documentation/scsi/st.txt |7 ++-
 drivers/scsi/st.c |   43 +++
 2 files changed, 49 insertions(+), 1 deletion(-)

Index: linux-2.6.25-rc2-q/drivers/scsi/st.c
===
--- linux-2.6.25-rc2-q.orig/drivers/scsi/st.c
+++ linux-2.6.25-rc2-q/drivers/scsi/st.c
@@ -4365,6 +4365,46 @@ static ssize_t st_defcompression_show(st
 
 CLASS_DEVICE_ATTR(default_compression, S_IRUGO, st_defcompression_show, NULL);
 
+static ssize_t st_options_show(struct class_device *class_dev, char *buf)
+{
+   struct st_modedef *STm = (struct st_modedef 
*)class_get_devdata(class_dev);
+   struct scsi_tape *STp;
+   int i, j, options;
+   ssize_t l = 0;
+
+   for (i=0; i  st_dev_max; i++) {
+   for (j=0; j  ST_NBR_MODES; j++)
+   if (scsi_tapes[i]-modes[j] == STm)
+   break;
+   if (j  ST_NBR_MODES)
+   break;
+   }
+   if (i == st_dev_max)
+   return 0;  /* should never happen */
+
+   STp = scsi_tapes[i];
+
+   options = STm-do_buffer_writes ? MT_ST_BUFFER_WRITES : 0;
+   options |= STm-do_async_writes ? MT_ST_ASYNC_WRITES : 0;
+   options |= STm-do_read_ahead ? MT_ST_READ_AHEAD : 0;
+   DEB( options |= debugging ? MT_ST_DEBUGGING : 0 );
+   options |= STp-two_fm ? MT_ST_TWO_FM : 0;
+   options |= STp-fast_mteom ? MT_ST_FAST_MTEOM : 0;
+   options |= STm-defaults_for_writes ? MT_ST_DEF_WRITES : 0;
+   options |= STp-can_bsr ? MT_ST_CAN_BSR : 0;
+   options |= STp-omit_blklims ? MT_ST_NO_BLKLIMS : 0;
+   options |= STp-can_partitions ? MT_ST_CAN_PARTITIONS : 0;
+   options |= STp-scsi2_logical ? MT_ST_SCSI2LOGICAL : 0;
+   options |= STm-sysv ? MT_ST_SYSV : 0;
+   options |= STp-immediate ? MT_ST_NOWAIT : 0;
+   options |= STp-sili ? MT_ST_SILI : 0;
+
+   l = snprintf(buf, PAGE_SIZE, 0x%08x\n, options);
+   return l;
+}
+
+CLASS_DEVICE_ATTR(options, S_IRUGO, st_options_show, NULL);
+
 static int do_create_class_files(struct scsi_tape *STp, int dev_num, int mode)
 {
int i, rew, error;
@@ -4402,6 +4442,9 @@ static int do_create_class_files(struct 
error = class_device_create_file(st_class_member,
class_device_attr_default_compression);
if (error) goto out;
+   error = class_device_create_file(st_class_member,
+   class_device_attr_options);
+   if (error) goto out;
 
if (mode == 0  rew == 0) {
error = 
sysfs_create_link(STp-device-sdev_gendev.kobj,
Index: linux-2.6.25-rc2-q/Documentation/scsi/st.txt
===
--- linux-2.6.25-rc2-q.orig/Documentation/scsi/st.txt
+++ linux-2.6.25-rc2-q/Documentation/scsi/st.txt
@@ -2,7 +2,7 @@ This file contains brief information abo
 The driver is currently maintained by Kai Mäkisara (email
 [EMAIL PROTECTED])
 
-Last modified: Thu Feb 21 21:54:16 2008 by kai.makisara
+Last modified: Sun Feb 24 21:59:07 2008 by kai.makisara
 
 
 BASICS
@@ -133,6 +133,11 @@ the defaults set by the user. The value 
 file 'dev' contains the device numbers corresponding to this device. The links
 'device' and 'driver' point to the SCSI device and driver entries.
 
+Each directory acontains also the entry 'options' which shows the currently
+enabled driver and mode options. The value in the file is a bit mask where the
+bit definitions are the same as those used with MTSETDRVBUFFER in setting the
+options.
+
 A link named 'tape' is made from the SCSI device directory to the class
 directory corresponding to the mode 0 auto-rewind device (e.g., st0). 
 

Re: (fwd) Bug#11922: I/O error on blank tapes

[Kai Makisara]

On Tue, 5 Feb 2008, Kai Makisara wrote:

 On Mon, 4 Feb 2008, James Bottomley wrote:
 
  
  On Mon, 2008-02-04 at 22:28 +0100, Borislav Petkov wrote:
   On Mon, Feb 04, 2008 at 03:22:06PM +0100, maximilian attems wrote:
   
   (Added Bart to CC)
   
hello borislav,

...
This does still occur with 2.6.22; with a blank tape in my HP DDS-4 
drive:

$ tar tzvf /dev/nst0
tar: /dev/nst0: Cannot read: Input/output error
  
  That's a SCSI tape, not an IDE one.  I cc'd the SCSI list
  
 This is not a bug, it is a feature. There is _nothing_ on the tape and if 
 you try to read something, you get an error. The same thing applies to 
 reading after the last filemark. Note that after writing a filemark at the 
 beginning of the tape, the situation is different. Now there is a file and 
 the normal EOF semantics apply although there still is no data.
 
 I admit that the error return could be more descriptive but the st driver 
 tries to be compatible with other Unices.
 
 The behavior can be changed if Linux does not match other Unices. I don't 
 remember if I have tested just this with other Unices. I will try to test 
 this with Tru64 tomorrow. If anyone has data on other Unices, it would be 
 helpful.
 
None of our Tru64 boxes have a user-accessible tape drive any more. 
However, I have been able to test with a Solaris box. The behavior there 
matches the Linux behavior: blank tape - i/o error when trying to read.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: (fwd) Bug#11922: I/O error on blank tapes

[Kai Makisara]

On Mon, 4 Feb 2008, James Bottomley wrote:

 
 On Mon, 2008-02-04 at 22:28 +0100, Borislav Petkov wrote:
  On Mon, Feb 04, 2008 at 03:22:06PM +0100, maximilian attems wrote:
  
  (Added Bart to CC)
  
   hello borislav,
   
   may i forward you that *old* Debian kernel bug,
   have seen you working on ide-tape:
   http://bugs.debian.org/11922
   no we don't carry any ide patches anymore.
   
   maybe you've already fixed it in latest?
   
   thanks
   
   -- 
   maks
   
   - Forwarded message from Stephen Kitt [EMAIL PROTECTED] -
   
   Subject: Bug#11922: I/O error on blank tapes
   Date: Sat, 1 Dec 2007 19:06:18 +0100
   From: Stephen Kitt [EMAIL PROTECTED]
   To: [EMAIL PROTECTED]
   
   Hi,
   
   This does still occur with 2.6.22; with a blank tape in my HP DDS-4 drive:
   
   $ tar tzvf /dev/nst0
   tar: /dev/nst0: Cannot read: Input/output error
 
 That's a SCSI tape, not an IDE one.  I cc'd the SCSI list
 
This is not a bug, it is a feature. There is _nothing_ on the tape and if 
you try to read something, you get an error. The same thing applies to 
reading after the last filemark. Note that after writing a filemark at the 
beginning of the tape, the situation is different. Now there is a file and 
the normal EOF semantics apply although there still is no data.

I admit that the error return could be more descriptive but the st driver 
tries to be compatible with other Unices.

The behavior can be changed if Linux does not match other Unices. I don't 
remember if I have tested just this with other Unices. I will try to test 
this with Tru64 tomorrow. If anyone has data on other Unices, it would be 
helpful.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] SCSI st : convert to unlocked_ioctl

[Kai Makisara]

Convert st to unlocked_ioctl. The necessary locking was already in place.

Signed-off-by: Kai Makisara [EMAIL PROTECTED]
---
The patch is against 2.6.24-rc8.

 drivers/scsi/st.c |9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

--- linux-2.6/drivers/scsi/st.c 2007-12-20 18:26:03.0 +0200
+++ linux-2.6-rc8-test/drivers/scsi/st.c2008-01-17 21:49:14.0 
+0200
@@ -9,7 +9,7 @@
Steve Hirsch, Andreas Koppenhofer, Michael Leodolter, Eyal Lebedinsky,
Michael Schaefer, Jorg Weule, and Eric Youngdale.
 
-   Copyright 1992 - 2007 Kai Makisara
+   Copyright 1992 - 2008 Kai Makisara
email [EMAIL PROTECTED]
 
Some small formal changes - aeb, 950809
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
  */
 
-static const char *verstr = 20070203;
+static const char *verstr = 20080117;
 
 #include linux/module.h
 
@@ -3214,8 +3214,7 @@ static int partition_tape(struct scsi_ta
 
 
 /* The ioctl command */
-static int st_ioctl(struct inode *inode, struct file *file,
-   unsigned int cmd_in, unsigned long arg)
+static long st_ioctl(struct file *file, unsigned int cmd_in, unsigned long arg)
 {
int i, cmd_nr, cmd_type, bt;
int retval = 0;
@@ -3870,7 +3869,7 @@ static const struct file_operations st_f
.owner =THIS_MODULE,
.read = st_read,
.write =st_write,
-   .ioctl =st_ioctl,
+   .unlocked_ioctl = st_ioctl,
 #ifdef CONFIG_COMPAT
.compat_ioctl = st_compat_ioctl,
 #endif
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: LTO-3 read performance issues

[Kai Makisara]

On Wed, 7 Nov 2007, James Pearson wrote:

 I have two LTO-3 (QUANTUM ULTRIUM 3) drives attached to a dual Adaptec U160
 controller (one per SCSI host) on a Dell PE2850 running a RHEL4 based kernel
 (2.6.9 based).
 
 I'm trying to read (with tar) LTO-3 tapes written on another system (possibly
 an SGI IRIX box), but I'm getting extremely variable read rates - from a few
 Kb/s to tens of Mb/s - while reading the same tape
 
 After a bit of trial and error, it looks like the tapes have been written in
 variable block mode with a block size of 16Kb
 
 To list the tapes, I need to set the block size to 0 (mt setblk 0) and run:
 
 tar tvfb $TAPE 32
 
 Running strace on the tar process shows that it does a number of read()'s then
 'sticks' on a read() for a number of seconds, and then does a burst of
 read()'s - the number of reads it does in these bursts and the time if waits
 on a particular read vary.
 
 My guess this is something to do the drive having to repositioning the tape
 between reads and breaking the tape streaming ...
 
The Quuntum LTO-3 sustained transfer rate is 68 MB/s. Compression does 
increase this. In order to avoid stopping the tape, your system should be 
able to able to process the data continuously at this rate. If this is not 
the case, you can't avoid the stops.

 I get the same issue on both drives with different tapes from the same source.
 
 I am using the default st module options and not doing anything other than
 using 'mt setblk 0'.
 
 Is there anything I can do to get a decent, sustained read rate from these
 tapes?
 
If your system can process the data fast enough and the tape blocks are of 
the same size, you can try using fixed block mode and buffered transfers. 
This allows the driver to use larger SCSI reads. You have to load the st 
module with parameter buffer_kbs=xxx, where xxx is, for instance, 1024 (1 
MB buffer). You also have to disable direct i/o with the module parameter
try_direct_io=0 (otherwise buffered transfers are disabled).

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/5] Use mutex instead of semaphore in the SCSI Tape driver

[Kai Makisara]

On Sun, 29 Jul 2007, Matthias Kaehlcke wrote:

 The SCSI Tape driver uses a semaphore as mutex. Use the mutex API
 instead of the (binary) semaphore.
 
 Signed-off-by: Matthias Kaehlcke [EMAIL PROTECTED]
 
Signed-off-by: Kai Makisara [EMAIL PROTECTED]

Thanks.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: aic7xxx: tape always rewinds since linux-2.6.15

[Kai Makisara]

On Tue, 17 Jul 2007, Frank Hempel wrote:

 Hello,
 
 I have expierenced a problem with the aic7xxx driver or the scsi tape support
 in the linux kernel.
 
 [Problem Description]
 Under linux kernels since 2.6.15 every action I issue to the tape drive (I
 mainly do this via mt from the cpio package) is done but additionaly to
 every mt call a tape rewind is done too. So for example positioning the tape
 at some particular position is impossible because after the positioning itself
 it is rewinded automagically.
 Under kernels before and including 2.6.14.7 this problem did not occur.
 
...
 [Attachments]
 kernel-config ... the relevant scsi-switches for all the
 kernels I used
 14.7.dmesg... the (hopefully) relevant dmesg output from
 2.6.14.7 kernel
 14.7.mt-status-after-boot ... the output of mt status after boot (tape
 was at block 0)
 14.7.mt-fsf-2.kern.log... the kernel output during mt fsf 2
 14.7.mt-status-after-mt-fsf-2 ... the output of mt status after mt fsf 2
 (file number is now *2*)

/dev/tape is link to /dev/nst0 or the environment variable TAPE has value 
/dev/nst0?

 15.dmesg  ... the (hopefully) relevant dmesg output from
 2.15 kernel
 15.mt-status-after-boot   ... the output of mt -f /dev/st0 status after
 boot (tape was at block 0)
 15.mt-fsf-2.kern.log  ... the kernel output during mt -f /dev/st0 fsf
 2
The device /dev/st0 is usually the auto-rewind device and, by definition, 
the tape is rewound after the device is closed. You should use /dev/nst0 
if you don't want the tape to be rewound after close.

If the tape was not rewound before 2.6.15 when you used /dev/st0, that 
was a problem.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/33] SG table chaining support

[Kai Makisara]

On Mon, 16 Jul 2007, Martin K. Petersen wrote:

  John == John Stoffel [EMAIL PROTECTED] writes:
 
 John Will this help out tape drive performance at all?  I looked
 John through the patches quickly, esp the AIC7xxx stuff since that's
 John what I use, but nothing jumped out at me...
 
 Yes.  Most modern tape drives want a block size of 1MB or higher.
 With the old stack we'd be stuck at 512KB because the sg limitations
 caused us to come just short of 1MB...
 
Tape block sizes up to 16 MB have been possible for a very long time but 
this has required tuning of the block/scsi parameters. Very few people 
seem to have done this and the common (mis)belief seems to be that the 
tape block size limit has been 512 kB. It is good if this tuning is not
needed in future.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Linux tape drivers

[Kai Makisara]

On Tue, 3 Apr 2007, Andrew Morton wrote:

 
 (cc's added, with permission)
 
 On Tue, 3 Apr 2007 15:08:37 +0200
 Kern Sibbald [EMAIL PROTECTED] wrote:
 
  Hello,
  
  I am the project manager for Bacula, an Open Source network backup program 
  that runs on all popular OSes.  After your presentation at FOSDEM in 
  Febrary, 
  we briefly talked about Linux tape driver problems I am encountering, and 
  you 
  offered to put me in touch with the appropriate kernel developers.
  
  I would much appreciate any help in this.  Since the problems concern all 
  tape 
  drivers, I provide a very brief outline of what my would like to discuss.  
  First, I must mention that the Linux SCSI driver works perfectly fine with 
  Bacula, it is simply a question of possible improvements, under item 2 
  below.
  
  Issues for discussion:
  
  1. Bugs:
 a. Other than the OSST driver, apparently no IDE/SATA tape driver works
 with Bacula. I don't have such a drive (working on it), but from user
 reports, it appears to me that there are problems of permitting 
 variable length blocks, and more serious, when writing to the end of
 the tape, either the logical end of tape indicator is ignored, or 
  when
 it is encountered, all further I/O is prohibited -- including a 
  WEOF. 
 This makes reliable writing of multiple reel tapes impossible.
  
 By the way, these IDE/SATA drives work with Bacula using the same
 source code cross-compiled with GNU C++ on Linux, then run on Windows
 machines, so it is most likely a driver issue rather than anything in
 Bacula or the hardware.
  

Others have already answered this and I agree with their view. All of the 
tape drives seem to use the SSC command set or something close to that. 
One high-level driver should be enough to implement the user semantics.
 
Libata should be able to drive the SATA/IDE drives using and the drives 
are visible as SCSI devices in Linux. In future there should be no real 
need for ide-scsi. Probably very few people have tried libata with tapes 
and there may be some problems to fix. Someone should test this with 
real devices and report the problems back to libata maintainers.

  2. Usability of the current tape driver API (not bugs)
a. With the new O_NONBLOCK flag introduced in kernel 2.5.x, opening
a tape drive and finding out if a volume is mounted is much more 
complicated.  It is really inconvenient and required a lot more code
in prior kernels.  This should be an item for discussion.

The reasons for the change were:
1. To be compatible with the Unix standards, and
2. To be compatible with other Unix tape driver semantics.

Because of these reasons the changes should probably not be reversed but 
there may be something to improve in the implementation. Suggestions?

b. There is no simple way to determine if a tape is in a drive -- it is 
at least 20 or 30 lines of C code to do it right.

Why not use GMT_ONLINE() with MTIOCGET? The definition from the st man 
page is:

GMT_ONLINE(x): The last open() found the drive with a tape in place and 
ready for operation.

If it does not work correctly, it can be fixed. (Of course, if you want to 
see if a tape is in a drive but not loaded, it is more difficult.)

c. There is a real lack of information about any error condition
(read/write). One can probably get it via direct SCSI commands, but
why not through an ioctl.

This is true. The problem is what and how to tell the user so that the 
solution is satisfactory for a reasonable number of years. The solutions I 
have thought about have not passed this test. One alternative is to pass 
the latest sense data to the user but thinking about the different error 
conditions, this is a problematic choice. Here also suggestions from and 
discussion with real users is needed.

d. The same as c. above, but for all kinds of error information that 
 can be important -- particularly soft errors.  An ioctl() that 
  returns
 additional information could be very useful.  Currently we get it by
 using external SCSI programs such as mtx tapeinfo for TapeAlerts,
 and smartclt for errors.

Looking at what the current drives provide in log and what the kernel can 
log, I currently think that this information is best obtained with 
external tools, like you do. The number of things that can be asked from 
the drives is very large and defining a kernel interface is difficult. The 
set also is evolving. (User's suggestions are also here welcome :-)

One way to proceed might be to make a user space library that presents an 
API to users and implements it using SG_IO. If the API seems to stabilize 
and be suitable for kernel implementation, it could be later implemented 
in the kernel.

 e. Apparently only root can do the following (IMO, this is a bug or 
 programming oversight): 
  
   

Re: Linux tape drivers

[Kai Makisara]

On Wed, 4 Apr 2007, Kern Sibbald wrote:

 On Wednesday 04 April 2007 20:46, Kai Makisara wrote:
...
  c. There is a real lack of information about any error condition
  (read/write). One can probably get it via direct SCSI commands, 
 but
  why not through an ioctl.
  
  This is true. The problem is what and how to tell the user so that the 
  solution is satisfactory for a reasonable number of years. The solutions I 
  have thought about have not passed this test. One alternative is to pass 
  the latest sense data to the user but thinking about the different error 
  conditions, this is a problematic choice. Here also suggestions from and 
  discussion with real users is needed.
 
 Well, you have in front of you a real user.

Yes, you are one of the people I meant with real users.

   I was thinking about a new 
 ioctl() that would hopefully be simple, generic, and extensible enough that 
 it might be picked up by all Unixes. This is one case were even if it is a 
 Linux only solution, I will be really happy. Basically, when I get a -1 
 return from a read/write and ERRNO=EIO, I would like to be able to call an 
 ioctl() and get back some basic info about soft errors, hard errors, ...  If 
 you don't tell me that a new ioctl() is out of the question, I'll make a 
 proposal -- it will take me some time to put it together.
 
The general attitude seems to be against new ioctls but I don't want to 
rule out that. The mt interface already uses ioctls and they solve the 
serialisation problem nicely in this case.

Whatever the method of delivery will be, the most difficult question is 
the contents and format. I don't know any good examples from other 
systems, so it has to be Linux-specific (and hopefully adopted by others 
as you say). I will wait with interest for your suggestion.

...
   e. Apparently only root can do the following (IMO, this is a bug or 
   programming oversight): 

 struct mtop mt;
 mt.mt_op = MTSETDRVBUFFER;
 mt.mt_count = MT_ST_CLEARBOOLEANS;
 mt.mt_count |= MT_ST_FAST_MTEOM;
 ...
 ioctl(fd, MIOCTOP, mt);

  Typically Bacula runs tape daemon as non-root.  Is there a good 
 reason 
   why program that has write permission on the device cannot do 
 this
   ioctl() and is it possible to change this?

  This is not a bug or an oversight, it is a design decision. The idea is 
  that the system provides the same view of the tape to all users. This view 
  is defined by the system manager (probably with startup scripts). If each 
  user can change the behaviour, the next user never knows what to expect. 
 
 Well, I agree that all users should have the same view.  However, when Bacula 
 has the drive open, it has it open exclusively, so no other users can view 
 the drive. When Bacula opens the drive, it should be able to ensure that the 
 drive, agrees with Bacula's concept of the drive.  When Bacula closes the 
 drive, I have no problem if the OS resets values to the common view.  Today, 
 if Bacula is running as bacula:disk rather than root:root, it cannot make the 
 above changes, even if the disk group has write access to the drive.

OK. I have not thought about this properly but one possible solution might
be something like this:
- separate the current and permanent behaviour flags; the current flags 
  define the behaviour
- the permanent flags are copied into the current flags when the device is
  opened (or at some other well-defined point)
- the booleans set with MTSETDRVBUFFER change always the current flags; if
  the user has CAP_SYS_ADMIN, also the permanent flags are changed

This would enable anyone to change the behaviour for his/her own purposes 
but there would still be a stable system-defined behaviour. There already 
are attributes that have a current and default value like block size.

I can see several problems in the details of this suggestion but I will 
think about those. One is that if someone having CAP_SYS_ADMIN wants just 
a temporary change, it is not possible. A solution would to have a new 
op code for temporary changes but this would require explicit changes to 
programs. Comments are welcome.

...
  Strictly speaking MTSETDRVBUFFER does not need root privileges. It needs 
  CAP_SYS_ADMIN. (Now I see that the error message is misleading and must be 
  fixed.) Does using some other capability make your use easier but, at same 
  time, limit the access to this command to a selected group of users?
 
 Being relatively ignorant of the kernel, I wasn't aware of CAP_SYS_ADMIN.  I 
 now have a rough idea of what it is, I am unsure if another capability would 
 resolve the problem, because I'm not familar with how they are set.  
 
Yes, setting the capabilities may be a problem. I don't know how well the 
different distributions are able to handle these finer-grained rights. 
A software

Re: Fwd: kernel oops SCSI tape drive access kernel 2.6.18

[Kai Makisara]

On Mon, 12 Feb 2007, Oliver Paulus wrote:

 Hello,
 
 I get a kernel oops if I want to read from my SCSI HP tape (C5683A) drive -
 using tar command.
 
 Here my system information:
 1. Kernel: 2.6.18-3-vserver-amd64
 2. OS: Debian Etch
 
...
 I get a kernel oops if I want to access the /dev/st0 with tar (or mondorescue
 restore - because this program uses tar too). Here the oops info:
 Feb  8 20:15:35 hal9000 kernel: st0: Block limits 1 - 16777215 bytes.
 Feb  8 20:15:56 hal9000 kernel: Unable to handle kernel paging request at
 82bc8057b000 RIP:
 Feb  8 20:15:56 hal9000 kernel:  [8020748b] 
 kmem_cache_free+0x42/0x1bb
 Feb  8 20:15:56 hal9000 kernel: PGD 0
 Feb  8 20:15:56 hal9000 kernel: Oops:  [13] SMP
 Feb  8 20:15:56 hal9000 kernel: CPU 0
 Feb  8 20:15:56 hal9000 kernel: Modules linked in: usb_storage cdc_acm ipv6
 dm_snapshot dm_mirror dm_mod loop snd_hda_intel snd_hda_codec serio_raw 
 snd_pcm
 snd_timer snd soundcore snd_page_alloc pcspkr psmouse shpchp pci_hotplug st
 joydev evdev ext3 jbd mbcache raid1 md_mod ide_generic sd_mod ide_cd cdrom
 usbhid aic7xxx scsi_transport_spi via_rhine mii ehci_hcd sata_via via82cxxx
 uhci_hcd libata scsi_mod generic ide_core processor
 Feb  8 20:15:56 hal9000 kernel: Pid: 16448, comm: tar Not tainted
 2.6.18-3-vserver-amd64 #1
 Feb  8 20:15:56 hal9000 kernel: RIP: 0010:[8020748b]
 [8020748b] kmem_cache_free+0x42/0x1bb
...
 Would be great if you can help me. Is this an already known issue?
 
It may be a known issue. A somewhat similar oops was reported in the 
Bugzilla item 7919. See 
http://marc.theaimsgroup.com/?l=linux-scsim=117050154700255w=2
for more information and a patch. The patch is also in the current linux 
git tree.

Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bugme-new] [Bug 7919] New: Tape dies if wrong block size used

[Kai Makisara]

On Thu, 1 Feb 2007, Andrew Morton wrote:

 On Thu, 1 Feb 2007 15:34:29 -0800
 [EMAIL PROTECTED] wrote:
 
  http://bugzilla.kernel.org/show_bug.cgi?id=7919
  
 Summary: Tape dies if wrong block size used
  Kernel Version: 2.6.20-rc5
  Status: NEW
Severity: normal
   Owner: [EMAIL PROTECTED]
   Submitter: [EMAIL PROTECTED]
  
  
  Most recent kernel where this bug did *NOT* occur: 2.6.17.14
  
  Other Kernels Tested and Results:
  
  OK 2.6.15.7
  OK 2.6.16.37 
  OK 2.6.17.14 
  BAD 2.6.18.6
  BAD 2.6.18-1.2869.fc6
  BAD 2.6.19.2 +
  BAD 2.6.20-rc5
  
  NOTE: 2.6.18-1.2869.fc6 is a Fedora modified kernel, all others are from 
  kernel.org
  
...
  Steps to reproduce: 
  Get a Adaptec AHA-2940U/UW/D / AIC-7881U card and a tape drive,
  install a recent kernel
  set the tape block size - mt setblk 4096
  read from or write to tape using wrong block size - tar -b 7 -cvf /dev/tape 
  foo
 
Write does not trigger this bug because the driver refuses in fixed block 
mode writes that are not a multiple of the block size. Read does trigger 
it in my system.

The bug is not associated with any specific HBA. st tries to do direct i/o 
in fixed block mode with reads that are not a multiple of tape block size. 

The patch in this message fixes the st problem by switching to using the 
driver buffer up to the next close of the device file in fixed block mode 
if the user asks for a read like this.

I don't know why the bug has surfaced only after 2.6.17 although the st 
problem is old. There may be another bug in the block subsystem and this 
patch works around it. However, the patch fixes a problem in st and in 
this way it is a valid fix.

This patch may also fix the bug 7900.

The patch compiles and is lightly tested.

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6/drivers/scsi/st.c 2006-12-09 13:29:31.0 +0200
+++ linux-2.6.20-rc7-km/drivers/scsi/st.c   2007-02-03 12:52:05.0 
+0200
@@ -9,7 +9,7 @@
Steve Hirsch, Andreas Koppenhofer, Michael Leodolter, Eyal Lebedinsky,
Michael Schaefer, Jorg Weule, and Eric Youngdale.
 
-   Copyright 1992 - 2006 Kai Makisara
+   Copyright 1992 - 2007 Kai Makisara
email [EMAIL PROTECTED]
 
Some small formal changes - aeb, 950809
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
  */
 
-static const char *verstr = 20061107;
+static const char *verstr = 20070203;
 
 #include linux/module.h
 
@@ -1168,6 +1168,7 @@ static int st_open(struct inode *inode, 
STps = (STp-ps[i]);
STps-rw = ST_IDLE;
}
+   STp-try_dio_now = STp-try_dio;
STp-recover_count = 0;
DEB( STp-nbr_waits = STp-nbr_finished = 0;
 STp-nbr_requests = STp-nbr_dio = STp-nbr_pages = 
STp-nbr_combinable = 0; )
@@ -1400,9 +1401,9 @@ static int setup_buffering(struct scsi_t
struct st_buffer *STbp = STp-buffer;
 
if (is_read)
-   i = STp-try_dio  try_rdio;
+   i = STp-try_dio_now  try_rdio;
else
-   i = STp-try_dio  try_wdio;
+   i = STp-try_dio_now  try_wdio;
 
if (i  ((unsigned long)buf  queue_dma_alignment(
STp-device-request_queue)) == 0) {
@@ -1599,7 +1600,7 @@ st_write(struct file *filp, const char _
STm-do_async_writes  STps-eof  ST_EOM_OK;
 
if (STp-block_size != 0  STm-do_buffer_writes 
-   !(STp-try_dio  try_wdio)  STps-eof  ST_EOM_OK 
+   !(STp-try_dio_now  try_wdio)  STps-eof  ST_EOM_OK 
STbp-buffer_bytes  STbp-buffer_size) {
STp-dirty = 1;
/* Don't write a buffer that is not full enough. */
@@ -1769,7 +1770,7 @@ static long read_tape(struct scsi_tape *
if (STp-block_size == 0)
blks = bytes = count;
else {
-   if (!(STp-try_dio  try_rdio)  STm-do_read_ahead) {
+   if (!(STp-try_dio_now  try_rdio)  STm-do_read_ahead) {
blks = (STp-buffer)-buffer_blocks;
bytes = blks * STp-block_size;
} else {
@@ -1948,10 +1949,12 @@ st_read(struct file *filp, char __user *
goto out;
 
STm = (STp-modes[STp-current_mode]);
-   if (!(STm-do_read_ahead)  STp-block_size != 0 
-   (count % STp-block_size) != 0) {
-   retval = (-EINVAL); /* Read must be integral number of 
blocks */
-   goto out;
+   if (STp-block_size != 0  (count % STp-block_size) != 0) {
+   if (!STm-do_read_ahead) {
+   retval = (-EINVAL); /* Read must be integral number 
of blocks */
+   goto out;
+   }
+   STp-try_dio_now = 0;  /* Direct i/o can't handle split blocks 
*/
}
 
STps = (STp-ps

Re: [Bugme-new] [Bug 7900] New: Kernel OOPS when using tape drive (compiler bug?)

[Kai Makisara]

On Mon, 29 Jan 2007, Andrew Morton wrote:

 On Mon, 29 Jan 2007 17:39:25 -0800
 [EMAIL PROTECTED] wrote:
 
  http://bugzilla.kernel.org/show_bug.cgi?id=7900
  
 Summary: Kernel OOPS when using tape drive (compiler bug?)
  Kernel Version: 2.6.20-rc5
  Status: NEW
Severity: high
   Owner: [EMAIL PROTECTED]
   Submitter: [EMAIL PROTECTED]
  
  
  Most recent kernel where this bug did *NOT* occur:
  Distribution: OpenSUSE 10.2 x86
  Hardware Environment: 
  SCSI controller LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual
  Ultra320 SCSI (rev 07), tape streamer Datastor LTO1 (don't know the exact 
  model)
  
  Software Environment: OpenSUSE 10.2 x86, mondoarchive, mondorescue
  
  Problem Description:
  Attempt to restore data from a tape drive results in the following Oops and
  subsequent functinality loss of the strreamer. Writing to the tape, however,
  worked fine. Or at least mondoarchive haven't complained.
  
I don't know how mondoarchive uses the tape. The bug 7900 looked somewhat 
similar. It might be useful to try the patch provided for that bug (in 
Bugzilla and linux-scsi).

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bugme-new] [Bug 7919] New: Tape dies if wrong block size used

[Kai Makisara]

On Sat, 3 Feb 2007, James Bottomley wrote:

 On Sat, 2007-02-03 at 13:21 +0200, Kai Makisara wrote:
  This patch may also fix the bug 7900.
  
  The patch compiles and is lightly tested.
 
 We can give it a spin in scsi-misc ... do you want me to hold off from
 sending it upstream with the scsi-misc tree when 2.6.20 is declared?
 
You can send it upstream after 2.6.20 is out. I am actually very happy 
with the patch. Conceptually it is very simple and based on mechanisms 
existing in the driver. In addition to fixing the bug in this report, it 
removes the last difference in user space sematics between direct i/o and 
using the driver buffer. (No documentation change needed because 
Documentation/scsi/st.txt has not mentioned this difference ;-)

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bugme-new] [Bug 7864] New: A MTIOCTOP/MTWEOF within the early warning will cause the file number to be incorrect

[Kai Makisara]

On Wed, 24 Jan 2007, Andrew Morton wrote:

 On Mon, 22 Jan 2007 13:07:20 -0800
 [EMAIL PROTECTED] wrote:
 
  http://bugzilla.kernel.org/show_bug.cgi?id=7864
  
 Summary: A MTIOCTOP/MTWEOF within the early warning will cause
  the file number to be incorrect
  Kernel Version: 2.6.19.2
  Status: NEW
Severity: low
   Owner: [EMAIL PROTECTED]
   Submitter: [EMAIL PROTECTED]
  
  
  Write records to a SCSI tape until a write fails with a ENOSPC (you have 
  reached
  early warning.
  Now perform a:
 struct mtget before, after;
 ioctl(fd, MTIOCGET, before);
 struct mtop mtop = { MTWEOF, 1 };
 ioctl(fd, MTIOCTOP, mtop);
 ioctl(fd, MTIOCGET, after);
  
  Check the value of mt_fileno in the before and after structures. Notice the
  after is 2 greater then the before.
  
  The problem appears to be in the block of code starting at line 2817 in 
  st.c.
  This block is entered because the drive did return a CHECK CONDITION with NO
  SENSE and the SENSE_EOM bit set. At lines 2824/5 the fileno is incremented. 
  But
  it has already been increased by the number of filemarks requested by the
  MTIOCTOP. I believe that the residue count in the sense data should be
  subtracted from fileno, not a increment as is done.
  
 
 Thanks.  Could you please send us a tested patch to fix these things, as
 per http://www.zip.com.au/~akpm/linux/patches/stuff/tpp.txt ?
 
The analysis is basically correct and explains the bug. According to the 
SCSI standards, the sense code is NO SENSE or RECOVERED ERROR in case 
writing filemark(s) succeeds. If it fails (partly or completely) the sense 
code is VOLUME OVERFLOW. The patch below is tested to fix the case when 
one filemark is successfully written after the EOM early warning. It 
should also fix the case at real EOM but this has not been tested.

Carl, thanks for reporting the bug and providing the analysis for the fix.

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6/drivers/scsi/st.c 2006-12-09 13:29:31.0 +0200
+++ linux-2.6.20-rc6-km/drivers/scsi/st.c   2007-01-25 22:51:35.0 
+0200
@@ -2816,15 +2816,18 @@ static int st_int_ioctl(struct scsi_tape
 
if (cmd_in == MTWEOF 
cmdstatp-have_sense 
-   (cmdstatp-flags  SENSE_EOM) 
-   (cmdstatp-sense_hdr.sense_key == NO_SENSE ||
-cmdstatp-sense_hdr.sense_key == RECOVERED_ERROR) 
-   undone == 0) {
-   ioctl_result = 0;   /* EOF written successfully at 
EOM */
-   if (fileno = 0)
-   fileno++;
+   (cmdstatp-flags  SENSE_EOM)) {
+   if (cmdstatp-sense_hdr.sense_key == NO_SENSE ||
+   cmdstatp-sense_hdr.sense_key == RECOVERED_ERROR) {
+   ioctl_result = 0;   /* EOF(s) written 
successfully at EOM */
+   STps-eof = ST_NOEOF;
+   } else {  /* Writing EOF(s) failed */
+   if (fileno = 0)
+   fileno -= undone;
+   if (undone  arg)
+   STps-eof = ST_NOEOF;
+   }
STps-drv_file = fileno;
-   STps-eof = ST_NOEOF;
} else if ((cmd_in == MTFSF) || (cmd_in == MTFSFM)) {
if (fileno = 0)
STps-drv_file = fileno - undone;
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Bugme-new] [Bug 7717] New: Tapes unreadable after unload

[Kai Makisara]

On Wed, 20 Dec 2006, Andrew Morton wrote:

 
 (please respond via emailed reply-to-all)
 
 On Wed, 20 Dec 2006 02:31:18 -0800
 [EMAIL PROTECTED] wrote:
 
  http://bugzilla.kernel.org/show_bug.cgi?id=7717
  
 Summary: Tapes unreadable after unload
  Kernel Version: tested 2.6.17 through 2.6.29 with various results
 
 I assume that means 2.6.17 and 2.6.19 failed.
 
  Status: NEW
Severity: blocking
   Owner: [EMAIL PROTECTED]
   Submitter: [EMAIL PROTECTED]
  
  
  Most recent kernel where this bug did *NOT* occur: 2.6.19
 
 And that means that 2.6.19 did not fail.
 
 Please clarify.
 
  Distribution: Slackware/Fedora
  Hardware Environment: HP 530, HP Dl380
  Software Environment: tar, mtx
  Problem Description: Doing a tar backup works well. Data is written and is
  readable. Binary restore comparison is ok. After unload/load the tape 
  produces
  Input/output errors. The tape is unreadable with working kernels too. Any
  attempt to restore data from faulty tapes were in vain until now. 
  
  Steps to reproduce:
  tar cvf /dev/st0 /path/to/something
  mtx -f /dev/sg1 unload
  mtx -f /dev/sg1 load X
  tar tvf /dev/st0
  

I tried this with 2.6.19 and the tape was readable. Unfortunately I don't 
have any older kernels around to test. I have not heard about any problem 
like this.

Note that the data becomes unreadable after the tape is touched by mtx 
through sg device, i.e, bypassing st.

  tar output:
  tar: /dev/st0: Cannot read: Input/output error
  tar: At beginning of tape, quitting now
  tar: Error is not recoverable: exiting now
  
Do you find any messages with dmesg or in system log?

  All read() attempts fail after 42 bytes. (I can attach those 42 bytes if 
  they
  are of any help).
 
Interesting byte count :-)

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fw: [Bugme-new] [Bug 7610] New: sense key error when sending even number of bytes to /dev/st0

[Kai Makisara]

On Fri, 1 Dec 2006, Andrew Morton wrote:

 
 
 Begin forwarded message:
 
 Date: Fri, 1 Dec 2006 08:40:16 -0800
 From: [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Subject: [Bugme-new] [Bug 7610] New: sense key error when sending even number 
 of bytes to /dev/st0
 
 
 http://bugzilla.kernel.org/show_bug.cgi?id=7610
 
Summary: sense key error when sending even number of bytes to
 /dev/st0
 Kernel Version: 2.6.19
 Status: NEW
   Severity: normal
  Owner: [EMAIL PROTECTED]
  Submitter: [EMAIL PROTECTED]
 
 
 Most recent kernel where this bug did *NOT* occur: unknown
 Distribution: Debian, but using kernel from kernel.org
 Hardware Environment: Dell Poweredge 2800 Xeon, Quantum DLT VS160 tape drive, 
 scsi1 : Adaptec AIC7XXX EISA/VLB/PCI SCSI HBA DRIVER, Rev 7.0
 Adaptec 3960D Ultra160 SCSI adapter
 aic7899: Ultra160 Wide Channel B, SCSI Id=7, 32/253 SCBs
 Software Environment: 
 Gnu C  3.3.5
 Gnu make   3.80
 binutils   2.15
 util-linux 2.12p
 mount  2.12p
 module-init-tools  3.2.2
 e2fsprogs  1.37
 nfs-utils  1.0.6
 Linux C Library2.3.6
 Dynamic linker (ldd)   2.3.6
 Procps 3.2.1
 Net-tools  1.60
 Console-tools  0.2.3
 Sh-utils   5.2.1
 udev   088
 Modules Loaded st parport_pc parport
 
 Problem Description: 
 When sending an even number of bytes to /dev/st0, I get the following error in
 /var/log/kernel:
 
 Nov 30 16:13:47 mugsy kernel: st0: Current: sense key: Illegal Request
 Nov 30 16:13:47 mugsy kernel: Additional sense: Invalid field in cdb
 
 Steps to reproduce:
 this will write to the tape:
 
 echo A  /dev/st0
 echo ABC  /dev/st0
 
 this will not write to the tape and will produce the kernel error:
 
 echo AB  /dev/st0
 echo ABCD  /dev/st0
 
I think this is not a bug but a user error.

Firstly, the report should say that writing odd number of bytes does not 
work (echo adds newline to the data).

The reason for the problem is probably that the drive in question does not 
support writing blocks with odd number of bytes. I just tested the echo 
commands with my drive and all echos succeeded (kernel 2.6.19).

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Possible bug in scsi_lib.c:scsi_req_map_sg()

[Kai Makisara]

On Mon, 27 Nov 2006, Mike Christie wrote:

 Mike Christie wrote:
  Boaz Harrosh wrote:
  Playing with some tests which I admit are not 100% orthodox I have
  stumbled upon a bug that raises a serious question:
 
  In the call to scsi_execute_async() in the use_sg case, must the
  scatterlist* (pointed to by buffer) map a buffer that's contiguous in
  virtual memory or is it allowed to map disjoint segments of memory?
  
  I thought they were continguous. I think James has said before that they
  can be disjoint. When we converted sg it did not look like sg or st
  supported disjoint. The main non dio path used a buffer from
  get_free_pages so I thought that would always be contiguous. The dio
  path then always set the first sg offset, but the rest it set to zero.
 
 And the len is set to page size for the middle entries too.
 
 But for the non DIO st path we can end up with some middle sg entires
 that are not a full page so that code in scsi_execute_async is broken
 for that.

I don't think so. Only the last sg entry may have length != n * (page size) 
when the driver buffer is used.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RESEND 2/2] SCSI: add scsi_device-retries

[Kai Makisara]

On Sun, 19 Nov 2006, Tejun Heo wrote:

 Add scsi_device-retries to provide generic control over command
 retries, which is very similar to sdev-timeout.  The initial value is
 -1 and high level driver is free to override on attach if negative.
 Note that -1 has the same effect as 0 (no retry) and signifies that
 it's the default value.  As with sdev-timeout, sdev-retries is
 exported under the device sysfs node.
 
 All high level drivers are converted to use it for retry value.
 However, there are exceptions.
 
 * sd_sync_cache() and sr::get_sectorsize() loop three more times
   around normal command execution on failure.
 
 * st uses three retry limits - MAX_RETRIES, MAX_WRITE_RETRIES and
   MAX_READY_RETRIES, which are all zero.  This patch only converts
   MAX_RETRIES to sdev-retries.  Defining WRITE and READY retries in
   terms of sdev-retries would make more sense.
 
I am neither acking nor naking this now. The patch does not change st 
behavior but moves part of the retry strategy out of the driver. (Which is 
also good because it makes one of the retry limits user changeable.)

Combining all retry counts is something that may not be a good thing. 
Below is justification why st has three different retry limits.

For some devices one number of retries is not perfect for all functions.

The firmware of tape devices usually retries quite thoroughly before 
returning error. This is why the default zero applies to most cases.

MAX_WRITE_RETRIES is separate from MAX_RETRIES because a plausible 
strategy might be (maybe not any more) to allow no retries for write but 
allow retries for reading and repositioning. The writes should fail 
directly so that the minimum number of marginal errors will be written. 
The reads can be retried so that even marginal data can be recovered. 
(Note that this may lead to tape positioning errors and may not be a 
good strategy in all cases.)

MAX_READY_RETRIES is used for commands that don't move the tape. This is a 
situation where retrying is probably harmless.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] SCSI tape signed/unsigned fix

[Kai Makisara]

This patch fixes in st.c the bug in the signed/unsigned int comparison
reported by Doug Gilbert and fixed by him in sg.c (see [PATCH] sg direct
io/mmap oops). Doug fixed the comparison in sg.c. This fix for st.c does not
touch the comparison but makes both arguments signed to remove the
problem. The new code is adapted from linux/fs/bio.c.

Doug's fix is correct and simple, no question about it. I just have learned to
hate these signed/unsigned comparisons where they can be avoided (having
originally created this bug is one more reason ;-)

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6.13/drivers/scsi/st.c  2005-08-29 21:04:57.0 +0300
+++ linux-2.6.13-k1/drivers/scsi/st.c   2005-08-30 21:35:14.0 +0300
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
  */
 
-static char *verstr = 20050501;
+static char *verstr = 20050830;
 
 #include linux/module.h
 
@@ -4348,12 +4348,12 @@ static int st_map_user_pages(struct scat
 static int sgl_map_user_pages(struct scatterlist *sgl, const unsigned int 
max_pages, 
  unsigned long uaddr, size_t count, int rw)
 {
+   unsigned long end = (uaddr + count + PAGE_SIZE - 1)  PAGE_SHIFT;
+   unsigned long start = uaddr  PAGE_SHIFT;
+   const int nr_pages = end - start;
int res, i, j;
-   unsigned int nr_pages;
struct page **pages;
 
-   nr_pages = ((uaddr  ~PAGE_MASK) + count + ~PAGE_MASK)  PAGE_SHIFT;
-
/* User attempted Overflow! */
if ((uaddr + count)  uaddr)
return -EINVAL;
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fw: [Bugme-new] [Bug 5117] New: Panic when accessing scsi-tapedrives with 4G-remap

[Kai Makisara]

On Fri, 26 Aug 2005, Kai Makisara wrote:

 On Thu, 25 Aug 2005, Andrew Morton wrote:
 
  
  
  Begin forwarded message:
  
  Date: Tue, 23 Aug 2005 12:53:38 -0700
  From: [EMAIL PROTECTED]
  To: [EMAIL PROTECTED]
  Subject: [Bugme-new] [Bug 5117] New: Panic when accessing scsi-tapedrives 
  with 4G-remap
  
  
   http://bugzilla.kernel.org/show_bug.cgi?id=5117
  
  Summary: Panic when accessing scsi-tapedrives with 4G-remap
   Kernel Version: 2.6.12.5
  
  
  Could this purely be a highmem problem?   Is the zero-copy DMA feature of
  st.c known to work OK with x86 highmem?
 
 It is _not_ known _not_ to work ;-) I.e., I have received neither any 
 success nor any failure reports. I have not tested it because I don't have 
 any machine with enough memory (and I have not hacked a kernel to use 
 highmem with 512 MB of memory). I hope someone seeing this thread and 
 using highmem with tape can comment on this subject.
 
OK. I booted my test i386 machine with highmem=384m and did some tests. I 
also added a counter to st.c to count the highmem pages used for zero-copy 
DMA. I could not get dd to use highmem but with tar that succeeded. No 
extra messages were found in syslog during these tests.

BUT, at the same time I remembered that the system in the Bugzilla report 
was Athlon64 running FC3 x86_64. The x86_64 kernel does not have highmem.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fw: [Bugme-new] [Bug 5117] New: Panic when accessing scsi-tapedrives with 4G-remap

[Kai Makisara]

On Thu, 25 Aug 2005, Andrew Morton wrote:

 
 
 Begin forwarded message:
 
 Date: Tue, 23 Aug 2005 12:53:38 -0700
 From: [EMAIL PROTECTED]
 To: [EMAIL PROTECTED]
 Subject: [Bugme-new] [Bug 5117] New: Panic when accessing scsi-tapedrives 
 with 4G-remap
 
 
  http://bugzilla.kernel.org/show_bug.cgi?id=5117
 
 Summary: Panic when accessing scsi-tapedrives with 4G-remap
  Kernel Version: 2.6.12.5
 
 
 Could this purely be a highmem problem?   Is the zero-copy DMA feature of
 st.c known to work OK with x86 highmem?

It is _not_ known _not_ to work ;-) I.e., I have received neither any 
success nor any failure reports. I have not tested it because I don't have 
any machine with enough memory (and I have not hacked a kernel to use 
highmem with 512 MB of memory). I hope someone seeing this thread and 
using highmem with tape can comment on this subject.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Fix SCSI tape oops at module removal

[Kai Makisara]

Removing the SCSI tape module results in an oops in class_device_destroy if
any devices are present. The patch at the end of this message fixes the bug
by moving class_destroy() later in exit_st() so that the class still exists
when devices are removed. (The bug is old but class_simple_device_remove() did
nothing when the class did not exist.)

The patch also fixes a class leak in init_st() error path.

I would like to get this into 2.6.13 but it may be too late?

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6.13-rc5/drivers/scsi/st.c  2005-06-30 21:27:51.0 +0300
+++ linux-2.6.13-rc5-k1/drivers/scsi/st.c   2005-08-02 11:40:45.0 
+0300
@@ -4149,12 +4149,10 @@ static int __init init_st(void)
do_create_driverfs_files();
return 0;
}
-   if (st_sysfs_class)
-   class_destroy(st_sysfs_class);
unregister_chrdev_region(MKDEV(SCSI_TAPE_MAJOR, 0),
-
 ST_MAX_TAPE_ENTRIES);
}
+   class_destroy(st_sysfs_class);
 
printk(KERN_ERR Unable to get major %d for SCSI tapes\n, 
SCSI_TAPE_MAJOR);
return 1;
@@ -4162,13 +4160,11 @@ static int __init init_st(void)
 
 static void __exit exit_st(void)
 {
-   if (st_sysfs_class)
-   class_destroy(st_sysfs_class);
-   st_sysfs_class = NULL;
do_remove_driverfs_files();
scsi_unregister_driver(st_template.gendrv);
unregister_chrdev_region(MKDEV(SCSI_TAPE_MAJOR, 0),
 ST_MAX_TAPE_ENTRIES);
+   class_destroy(st_sysfs_class);
kfree(scsi_tapes);
printk(KERN_INFO st: Unloaded.\n);
 }
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH] drivers/scsi/st.c: add reference count and related fixes

[Kai Makisara]

I have rediffed the patch against 2.6.13-rc5, done a couple of cosmetic
cleanups, and run some tests.  Brian King has acknowledged that it fixes the
problems he has seen. Seems mature enough for inclusion into 2.6.14 (or
later)?

Nate's explanation of the changes:

I've attached patches against 2.6.13rc2. These are basically identical
to my earlier patches, as I found that all issues I'd seen in earlier
kernels still existed in this kernel.

To summarize, the changes are: (more details in my original email)

- add a kref to the scsi_tape structure, and associate reference
counting stuff

- set sr_request-end_io = blk_end_sync_rq so we get notified when an IO
is rejected when the device goes away

- check rq_status when IOs complete, else we don't know that IOs
rejected for a dead device in fact did not complete

- change last_SRpnt so it's set before an async IO is issued (in case
st_sleep_done is bypassed)

- fix a bogus use of last_SRpnt in st_chk_result


Nate Dailey
Stratus Technologies

Signed-off-by: Nate Dailey [EMAIL PROTECTED]
Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6.13-rc5-k1/drivers/scsi/st.c   2005-08-02 12:34:48.0 
+0300
+++ linux-2.6.13-rc5-k2/drivers/scsi/st.c   2005-08-02 12:32:21.0 
+0300
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
  */
 
-static char *verstr = 20050501;
+static char *verstr = 20050802;
 
 #include linux/module.h
 
@@ -219,6 +219,12 @@ static int switch_partition(struct scsi_
 
 static int st_int_ioctl(struct scsi_tape *, unsigned int, unsigned long);
 
+static void scsi_tape_release(struct kref *);
+
+#define to_scsi_tape(obj) container_of(obj, struct scsi_tape, kref)
+
+static DECLARE_MUTEX(st_ref_sem);
+
 
 #include osst_detect.h
 #ifndef SIGS_FROM_OSST
@@ -230,6 +236,46 @@ static int st_int_ioctl(struct scsi_tape
{OnStream, FW-, , osst}
 #endif
 
+static struct scsi_tape *scsi_tape_get(int dev)
+{
+   struct scsi_tape *STp = NULL;
+
+   down(st_ref_sem);
+   write_lock(st_dev_arr_lock);
+
+   if (dev  st_dev_max  scsi_tapes != NULL)
+   STp = scsi_tapes[dev];
+   if (!STp) goto out;
+
+   kref_get(STp-kref);
+
+   if (!STp-device)
+   goto out_put;
+
+   if (scsi_device_get(STp-device))
+   goto out_put;
+
+   goto out;
+
+out_put:
+   kref_put(STp-kref, scsi_tape_release);
+   STp = NULL;
+out:
+   write_unlock(st_dev_arr_lock);
+   up(st_ref_sem);
+   return STp;
+}
+
+static void scsi_tape_put(struct scsi_tape *STp)
+{
+   struct scsi_device *sdev = STp-device;
+
+   down(st_ref_sem);
+   kref_put(STp-kref, scsi_tape_release);
+   scsi_device_put(sdev);
+   up(st_ref_sem);
+}
+
 struct st_reject_data {
char *vendor;
char *model;
@@ -311,7 +357,7 @@ static int st_chk_result(struct scsi_tap
return 0;
 
cmdstatp = STp-buffer-cmdstat;
-   st_analyze_sense(STp-buffer-last_SRpnt, cmdstatp);
+   st_analyze_sense(SRpnt, cmdstatp);
 
if (cmdstatp-have_sense)
scode = STp-buffer-cmdstat.sense_hdr.sense_key;
@@ -399,10 +445,10 @@ static void st_sleep_done(struct scsi_cm
 
(STp-buffer)-cmdstat.midlevel_result = SCpnt-result;
SCpnt-request-rq_status = RQ_SCSI_DONE;
-   (STp-buffer)-last_SRpnt = SCpnt-sc_request;
DEB( STp-write_pending = 0; )
 
-   complete(SCpnt-request-waiting);
+   if (SCpnt-request-waiting)
+   complete(SCpnt-request-waiting);
 }
 
 /* Do the scsi command. Waits until command performed if do_wait is true.
@@ -412,8 +458,20 @@ static struct scsi_request *
 st_do_scsi(struct scsi_request * SRpnt, struct scsi_tape * STp, unsigned char 
*cmd,
   int bytes, int direction, int timeout, int retries, int do_wait)
 {
+   struct completion *waiting;
unsigned char *bp;
 
+   /* if async, make sure there's no command outstanding */
+   if (!do_wait  ((STp-buffer)-last_SRpnt)) {
+   printk(KERN_ERR %s: Async command already active.\n,
+  tape_name(STp));
+   if (signal_pending(current))
+   (STp-buffer)-syscall_result = (-EINTR);
+   else
+   (STp-buffer)-syscall_result = (-EBUSY);
+   return NULL;
+   }
+
if (SRpnt == NULL) {
SRpnt = scsi_allocate_request(STp-device, GFP_ATOMIC);
if (SRpnt == NULL) {
@@ -427,7 +485,13 @@ st_do_scsi(struct scsi_request * SRpnt, 
}
}
 
-   init_completion(STp-wait);
+   /* If async IO, set last_SRpnt. This ptr tells write_behind_check
+  which IO is outstanding. It's nulled out when the IO completes. */
+   if (!do_wait)
+   (STp-buffer)-last_SRpnt = SRpnt;
+
+   waiting = STp-wait;
+   init_completion(waiting);
SRpnt-sr_use_sg = STp-buffer-do_dio || (bytes

Re: [PATCH] drivers/scsi/st.c: add command and sense history

[Kai Makisara]

The following comments are based on looking at the patch, not testing it.

The tape devices do not provide much information about problems via the normal
API: mostly it just tells that something went wrong. The driver prints the
sense data to the kernel log but this information may not be accessible and/or
it is difficult to connect it to the specific problem at hand. Exporting the
sense data in some form is a better (although not perfect) method to give more
information in case of problems.

On Fri, 22 Jul 2005, Andy Stevens wrote:

 Hello SCSI tapers,
 
 Attached is a patch to add some functionality the st driver so that you can
 monitor the most recent SCSI command and sense buffer.  This info is made
 available via sysfs attributes.  I want to add these attributes in order to do
 better error processing in some custom tape software which I am writing.  This
 patch has been tested on Linux kernel 2.6.12.
 
 The new functionality is as follows:
 
 1) the most recent relevant SCSI command and sense are stored in hex format
 in:
 
 /sys/class/scsi_tape/stN/last_cmnd
 /sys/class/scsi_tape/stN/last_sense
 

Using two files is problematic: the user can't be sure that last_sense is
really related to last_cmnd. If they are in the same file, a single read could
enforce this. (The implementation does not contain the necessary locking. I
don't think it is necessary at this phase but the single file approach would
make atomicity possible if desired.)

 When I say relevant SCSI command, I mean: occasionally the most recent SCSI
 command is not the one of interest, e.g. when encountering an ILI, the driver
 will automatically backspace to the beginning of the block. However, the
 last_cmnd and last_sense will show the results of the read error, not the
 backspace.
 
This is the reason why this should be in the ULD. If only the most recent
data would be exported, it could be done at midlevel to benefit all SCSI
devices.

 2) The raw sense data is also decoded into English language for primary sense
 key, extended sense, and ILI/FM/EOM flags in these sysfs files:
 
 /sys/class/scsi_tape/stN/primary_sense
 /sys/class/scsi_tape/stN/extended_sense
 /sys/class/scsi_tape/stN/sense_flags
 
I don't think these are necessary. Anyway, the sysfs data should be either
single values or an array of values. The contents of the files must be
explained elsewhere (linux/Documentation/scsi/st.txt).

However, I would like to discuss exporting selected fields of the sense data 
instead
of the raw data as an alternative. This would make interpretation of the data
easier when devices supporting the descriptor based sense data start
appearing. The values exported could be: sense key, ASC, ASCQ, flags (FM, EOM,
ILI), information field. What do the users really need?

 Details of implementation:
 
 1) added data fields last_cmnd and last_sense to struct st_cmdstatus for
 storage of most recent command and sense data.  Removed unused last_cmnd and
 last_sense fields from struct scsi_tape.
 
 2) added routine st_record_last_command() to save command and sense data
 
 3) added calls to st_record_last_command() in appropriate locations in the
 code.
 
 4) added arg to st_int_ioctl() to indicate whether or not to record the SCSI
 command (necessary so that we can record the relevant command)
 
 5) added routines for sysfs attribute files
 
 This is my first time submitting a patch to the list, please go easy on me!!
 
Could you in future make patches from the linux source tree root using
'diff -up' (i.e., the path to st.c would be linux/drivers/scsi/st.c; the file
linux/Documentation/SubmittingPatches contains a lot of useful
information). Appending the diff instead of putting it into an attachment
would make commenting easier for some of us.

 --- st.c.orig 2005-06-17 15:48:29.0 -0400
 +++ st.c  2005-07-22 15:20:06.0 -0400
 @@ -17,7 +17,7 @@
 Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
   */
  
 -static char *verstr = 20050312;
 +static char *verstr = 20050706;
  
  #include linux/module.h
  
 @@ -215,7 +215,7 @@
  static int find_partition(struct scsi_tape *);
  static int switch_partition(struct scsi_tape *);
  
 -static int st_int_ioctl(struct scsi_tape *, unsigned int, unsigned long);
 +static int st_int_ioctl(struct scsi_tape *, unsigned int, unsigned long, 
 int);
  
  
  #include osst_detect.h
 @@ -264,6 +264,29 @@
   return tape-disk-disk_name;
  }
  
 +static void st_clear_last_command(struct scsi_tape *STp)
 +{
 + /* clear last command and last sense */
 + memset(STp-buffer-cmdstat.last_cmnd,0,MAX_COMMAND_SIZE);
 + memset(STp-buffer-cmdstat.last_sense,0,SCSI_SENSE_BUFFERSIZE);
 +}
 +
 +static void st_record_last_command(struct scsi_tape *STp, struct 
 scsi_request *SRpnt)
 +{
 + int i;
 +
 + st_clear_last_command(STp);
 + 

Not necessary.

 + /* make copy of last SCSI command */
 + for(i=0; iMAX_COMMAND_SIZE; i++){
 + 

RE: [PATCH] drivers/scsi/st.c: add reference count and related fixes

[Kai Makisara]

On Tue, 12 Jul 2005, Dailey, Nate wrote:

 I'll try porting my changes to a recent kernel, verify that the bugs I
 found are still bugs, and get back to you with a new patch.
 
 In the meantime, responses to your comments:
 
 - using a new kref in the scsi_tape structure: I've given this some
 thought, and I think using a new kref is defintely the best way to go.
 If nothing else, this is the same way sd and sr do reference counting...
 seems easier to maintain if the code in st is as similar as possible to
 these other drivers.
 
st is a character device and it should use the character device 
refcounting if convenient. However, I now agree that the new kref is the 
simplest solution.

 - my changes to how last_SRpnt is used: I'll add a comment documenting
 when last_SRpnt is valid (it's valid only for async IOs that have been
 issued, nulled out by write_behind_check when they complete)
 
Good.

 - Here's why I think the use of last_SRpnt in st_chk_result is bogus:
 one of the inputs to st_chk_result is struct scsi_request * SRpnt, and
 it seems like this is the SRpnt you want to use, rather than last_SRpnt.
 This input will be the command that st_do_scsi or write_behind_check
 just waited on. This might not cause any problems with the current st
 code, but it will definitely cause problems with the changes I made to
 last_SRpnt. Let me know if I'm wrong about this, though, and I can
 change my last_SRpnt code.
 
OK. I now see your point and it seems that the solution is simple: use the 
SRpnt argument of st_chk_result(). The reason why the function is using 
last_SRpnt is moving code from one place to another and not being careful 
enough to change everything (I knew that in that code last_SRpnt was valid 
always and so this did not look like an error ;-).

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] drivers/scsi/st.c: decrement in_use under the lock in st_open

[Kai Makisara]

On Thu, 21 Apr 2005, Nate Dailey wrote:

 Hi, this patch changes the err_out case in st_open to decrement STp-in_use 
 under the st_dev_arr_lock.
 
 I'm just basing this on looking at the code (I didn't run into an actual 
 problem here), but it seems like the right thing to do; st_open and 
 release hold the lock when adjusting in_use in other places.
 
I don't think it is necessary hold the lock when setting STp-in_use to 
zero. The problem is that the usage is not consistent in st. Either this  
patch should be included or the locking should be removed from st_release. 
I don't have any strong opinions about the best solution.

Thanks for pointing out this inconsistency.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH scsi-misc-2.6 08/08] scsi: fix hot unplug sequence

[Kai Makisara]

On Fri, 25 Mar 2005, James Bottomley wrote:

 On Fri, 2005-03-25 at 14:38 +0900, Tejun Heo wrote:
   We have users of scsi_do_req() other than scsi_wait_req() and they
  use different done() functions to do different things.  I've checked
  other done functions and none uses contents inside the passed
  scsi_cmnd, so using a dummy command should be okay with them.  Am I
  missing something here?
 
 Well ... the other users are supposed to be going away.  They're
 actually all coded wrongly in some way or other ... perhaps I should
 speed up the process.
 
I have seen you mention this several times now and I am getting more and 
more worried. The reason is that scsi_wait_req() is a synchronous 
interface and it does not allow a driver to do this:

- send a request
- do other useful things/let the user do useful work
- wait for completion before starting another request

I fully agree that doing done() correctly _is_ a problem, especially when 
the SCSI subsystem evolves and the high-level driver writers do not follow 
the development closely enough.

One solution to these problems would be to let the drivers still use 
scsi_do_req() and their own done() function, but create two 
(three) helpers:
- one to be called at the beginning of done(); it would do what needs to 
  be done here but lets the driver to do some special things of its own if
  necessary
- one to be called to wait for the request to finish
(- one to do scsi_ro_req() and the things necessary before these)

Having these helpers would isolate the user of the SCSI subsystem from the 
internals. scsi_wait_req() should call these functions and no additional 
maintenance would be needed for this additional asynchronous interface.

The current drivers may not do any work in done() that could not be done 
later but there is one patch pending where this happens: the st 
performance statistics patch needs to get the time stamp when the SCSI 
command is processed.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] SCSI tape fixes (new version): sense descriptor init, bsf-weof, blkno, debugging

[Kai Makisara]

On Tue, 15 Mar 2005, James Bottomley wrote:

 On Sat, 2005-03-12 at 19:04 +0200, Kai Makisara wrote:
  This is an updated version of the patch I sent March 7. The sense descriptor
  initialization has been made lighter.
  
  The patch at the end of this message applies to 2.6.11-bk7 + st descriptor 
  sense
  patch + st auto eof patch (i.e., st patches currently in scsi-misc-2.6). The
  patch fixes the following problems:
 
 I'm afraid this doesn't apply.  It looks like I've got an interim patch
 that's crept in somewhere in the tree.
 
 Could you take a look at scsi-misc-2.6 (or the patches on www.parisc-
 linux/~jejb/scsi_diffs) and see if you can construct a diff between
 where I currently am and where I should be?
 
The changelog shows that the patch below is included. It is the previous 
version of the patch in the message with this subject.

# ChangeSet
#   2005/03/15 10:37:07-06:00 [EMAIL PROTECTED] 
#   [PATCH] SCSI tape fixes: sense descriptor init, bsf-weof, blkno,

The patch at the end of this message should take you into the point where 
you can directly apply 2/2 etc. (If you think that the patch below is not 
worth the trouble, you can skip that. The other patches are not touching 
verstr and should then apply with small offset.)

Thanks,
Kai

This patch changes the sense descriptor initialization a little faster.

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6.11-kk1/drivers/scsi/st.c  2005-03-15 20:55:12.0 +0200
+++ linux-2.6.11-bk7-k2/drivers/scsi/st.c   2005-03-12 13:52:30.0 
+0200
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
  */
 
-static char *verstr = 20050307;
+static char *verstr = 20050312;
 
 #include linux/module.h
 
@@ -268,10 +268,11 @@ static void st_analyze_sense(struct scsi
const u8 *ucp;
const u8 *sense = SRpnt-sr_sense_buffer;
 
-   memset(s, 0, sizeof(struct st_cmdstatus));
s-have_sense = scsi_request_normalize_sense(SRpnt, s-sense_hdr);
+   s-flags = 0;
 
if (s-have_sense) {
+   s-deferred = 0;
s-remainder_valid =
scsi_get_sense_info_fld(sense, SCSI_SENSE_BUFFERSIZE, 
s-uremainder64);
switch (sense[0]  0x7f) {
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [patch 02/11] scsi/st: replace schedule_timeout() with msleep_interruptible()

[Kai Makisara]

On Sun, 6 Mar 2005 [EMAIL PROTECTED] wrote:

 
 
 Use msleep_interruptible() instead of
 schedule_timeout() to guarantee the task delays as expected.
 
This patch is already in the scsi-misc-2.6 tree merged into one of the st 
patches.

Thanks for the reminder and the patch.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] SCSI tape fixes: sense descriptor init, bsf-weof, blkno, debugging

[Kai Makisara]

The patch at the end of this message applies to 2.6.11 + st descriptor sense
patch + st auto eof patch (i.e., st patches currently in scsi-misc-2.6). The
patch fixes the following problems:

- the sense descriptor is cleared before filling
- BSF and BSFM are added to the commands causing automatic writing of filemark
  if the previous operation was write (tar expects this)
- the block number is set to unknown (-1) if spacing forward ends at BLANK CHECK
- debugging printout of spacing counts fixed to work also with 64-bit systems

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6.11-k1/drivers/scsi/st.c   2005-03-07 19:38:45.0 +0200
+++ linux-2.6.11-k2/drivers/scsi/st.c   2005-03-07 21:59:24.0 +0200
@@ -17,7 +17,7 @@
Last modified: 18-JAN-1998 Richard Gooch [EMAIL PROTECTED] Devfs support
  */
 
-static char *verstr = 20050213;
+static char *verstr = 20050307;
 
 #include linux/module.h
 
@@ -268,6 +268,7 @@ static void st_analyze_sense(struct scsi
const u8 *ucp;
const u8 *sense = SRpnt-sr_sense_buffer;
 
+   memset(s, 0, sizeof(struct st_cmdstatus));
s-have_sense = scsi_request_normalize_sense(SRpnt, s-sense_hdr);
 
if (s-have_sense) {
@@ -287,8 +288,6 @@ static void st_analyze_sense(struct scsi
ucp = scsi_sense_desc_find(sense, 
SCSI_SENSE_BUFFERSIZE, 4);
s-flags = ucp ? (ucp[3]  0xe0) : 0;
break;
-   default:
-   s-flags = 0;
}
}
 }
@@ -306,7 +305,7 @@ static int st_chk_result(struct scsi_tap
if (!result)
return 0;
 
-   cmdstatp = STp-buffer-cmdstat;
+   cmdstatp = STp-buffer-cmdstat;
st_analyze_sense(STp-buffer-last_SRpnt, cmdstatp);
 
if (cmdstatp-have_sense)
@@ -2407,6 +2406,22 @@ static int do_load_unload(struct scsi_ta
return retval;
 }
 
+#if DEBUG
+#define ST_DEB_FORWARD  0
+#define ST_DEB_BACKWARD 1
+static void deb_space_print(char *name, int direction, char *units, unsigned 
char *cmd)
+{
+   s32 sc;
+
+   sc = cmd[2]  0x80 ? 0xff00 : 0;
+   sc |= (cmd[2]  16) | (cmd[3]  8) | cmd[4];
+   if (direction)
+   sc = -sc;
+   printk(ST_DEB_MSG %s: Spacing tape %s over %d %s.\n, name,
+  direction ? backward : forward, sc, units);
+}
+#endif
+
 
 /* Internal ioctl function */
 static int st_int_ioctl(struct scsi_tape *STp, unsigned int cmd_in, unsigned 
long arg)
@@ -2445,8 +2460,7 @@ static int st_int_ioctl(struct scsi_tape
cmd[2] = (arg  16);
cmd[3] = (arg  8);
cmd[4] = arg;
-DEBC(printk(ST_DEB_MSG %s: Spacing tape forward over %d 
filemarks.\n,
-   name, cmd[2] * 65536 + cmd[3] * 256 + cmd[4]));
+DEBC(deb_space_print(name, ST_DEB_FORWARD, filemarks, cmd);)
if (fileno = 0)
fileno += arg;
blkno = 0;
@@ -2461,14 +2475,7 @@ static int st_int_ioctl(struct scsi_tape
cmd[2] = (ltmp  16);
cmd[3] = (ltmp  8);
cmd[4] = ltmp;
-DEBC(
- if (cmd[2]  0x80)
-   ltmp = 0xff00;
- ltmp = ltmp | (cmd[2]  16) | (cmd[3]  8) | cmd[4];
- printk(ST_DEB_MSG
-%s: Spacing tape backward over %ld filemarks.\n,
-name, (-ltmp));
-   )
+DEBC(deb_space_print(name, ST_DEB_BACKWARD, filemarks, cmd);)
if (fileno = 0)
fileno -= arg;
blkno = (-1);   /* We can't know the block number */
@@ -2480,8 +2487,7 @@ static int st_int_ioctl(struct scsi_tape
cmd[2] = (arg  16);
cmd[3] = (arg  8);
cmd[4] = arg;
-DEBC(printk(ST_DEB_MSG %s: Spacing tape forward %d 
blocks.\n, name,
-  cmd[2] * 65536 + cmd[3] * 256 + cmd[4]));
+DEBC(deb_space_print(name, ST_DEB_FORWARD, blocks, cmd);)
if (blkno = 0)
blkno += arg;
at_sm = (arg == 0);
@@ -2493,13 +2499,7 @@ static int st_int_ioctl(struct scsi_tape
cmd[2] = (ltmp  16);
cmd[3] = (ltmp  8);
cmd[4] = ltmp;
-DEBC(
- if (cmd[2]  0x80)
-  ltmp = 0xff00;
- ltmp = ltmp | (cmd[2]  16) | (cmd[3]  8) | cmd[4];
- printk(ST_DEB_MSG
-%s: Spacing tape backward %ld blocks.\n, name, 
(-ltmp));
-   )
+DEBC(deb_space_print(name, ST_DEB_BACKWARD, blocks, cmd);)
if (blkno = 0)
blkno -= arg;
at_sm = (arg == 0);
@@ -2510,8 +2510,7 @@ static int st_int_ioctl(struct

Re: 2.6.10-rc4 tape slow after switch Fusion MPT to Megaraid

[Kai Makisara]

On Fri, 25 Feb 2005, Gerhard Schneider wrote:

 On Thu, 2005-02-24 at 23:00 +0200, Kai Makisara wrote:
 
   
  Sounds strange but the following things come into my mind:
  - Check from the dmesg output and/or system logs what SCSI transfer 
  settings the tape drive and the SCSI adapter negotiate. The speed should 
  not change when you add the Megaraid controller. If you don't find the 
  value before adding it, you can check that the current value is at least 
  40 MB/s (probably more).
 Not so easy. The megaraid driver doesn't tell you that
 
I hope Seokmann can give you some information about this.

...
  - Enable debugging in the st driver (change DEBUG to 1 in st.c, recompile, 
  reload the module). In the normal case st should not print many debugging 
  message (some when the device is opened at some when it is closed).
 
 Now the full story:
 
 st: Version 20050213, fixed bufsize 32768, s/g segs 256
 Attached scsi tape st0 at scsi0, channel 0, id 6, lun 0
normal output cut
 st0: Number of r/w requests 127685, dio used in 127685, pages 383055

This shows that your application is not using very large block size (3 
pages, 10 kB?). You might get better throughput with larger block size (32 
kB or 64 kB is usually large enough). If the Megaraid command processing 
speed is not fast enough, this might explain the problem (not probable but 
I can't see any probable problems ;-)

The messages look OK. There is nothing that should not be there.

--
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Problem with tape drive: illegal seek

[Kai Makisara]

On Sat, 26 Feb 2005, Christian Iversen wrote:

 
 Whenever I use --verify for tar, I get problems. Specifically, I get 
 Illegal seek when tar gets to the verify phase. 
 
I looked at tar 1.13.25 sources and this looks like a tar bug: verify from 
tape does not work for the first file (and I am not sure about other 
files). After writing, tar tries to skip backwards over two filemarks. 
When writing the first file, there is no second filemark to skip over: 
this fails and gives you the error. If you do verify with a second call to 
tar (tar --compare) after skipping backwards with mt, this should work.

...
 Please note that I can restore the data just fine, if I use tar. If I try to 
 use dd, I get strange errors as well:
 
 [EMAIL PROTECTED]/tmp #dd if=/dev/tape of=dump
 dd: reading `/dev/tape': Cannot allocate memory
 0+0 records in
 0+0 records out
 0 bytes transferred in 3,699040 seconds (0 bytes/sec)
 
Guy already solved this problem. The message (Cannot allocate memory) is 
not very clear but this is what other Unices use.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 2.6.10-rc4 tape slow after switch Fusion MPT to Megaraid

[Kai Makisara]

On Wed, 23 Feb 2005, Gerhard Schneider wrote:

 
 Kernel: 2.6.10-rc4 w/ 3 tape patches from Kai Makisara
 
 SCSI: 53C1030 controller, tape drive is on one channel of the
 controller.
 
 Tape: Overland Changer w/ Seagate LTO-1 drive
 Typical transfer speed: 12MB/s
 
 After adding a Megaraid Zero Channel Raid Controller to that
 computer the transfer speed of the tape changed to 2.6 MB/s
 
 Disks on the other channel of the 53C1030 are NOT affected
 (disk performance OK).
 
 stinit.def has NOT been changed.
 
 No error messages.
 I didn't find any changable parameters for the Megaraid adapter
 for TAPE drives.
 
 What did I do wrong? How to provide you with additional data?
 
Sounds strange but the following things come into my mind:
- Check from the dmesg output and/or system logs what SCSI transfer 
settings the tape drive and the SCSI adapter negotiate. The speed should 
not change when you add the Megaraid controller. If you don't find the 
value before adding it, you can check that the current value is at least 
40 MB/s (probably more).
- Enable debugging in the st driver (change DEBUG to 1 in st.c, recompile, 
reload the module). In the normal case st should not print many debugging 
message (some when the device is opened at some when it is closed).
- Back out the tape patches and see if that improves the situation (these 
patches should not have any effect on speed and they did not in my tests 
but changes may contain bugs that show up in other environments ...)

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] SCSI tape: write filemark before rewind etc. when writing

[Kai Makisara]

The patch at the end of this message modifies the st write semantics in 
the following way: write a filemark before rewind, offline, or seek if the 
previous operation was write.

This semantics is specified on the man pages of some Unices and some 
software (e.g., cpio) seems to assume this. The change makes sure that the 
last file on the tape is properly terminated with a filemark and reading the 
file does not fail at the end.

The patch is against 2.6.11-rc4 + the descriptor based sense data patch I 
sent to linux-scsi on Feb 13.

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6.11-rc4-k1/drivers/scsi/st.c   2005-02-13 22:42:48.0 
+0200
+++ linux-2.6.11-rc4-k2/drivers/scsi/st.c   2005-02-20 22:22:51.0 
+0200
@@ -2727,6 +2727,8 @@ static int st_int_ioctl(struct scsi_tape
else if (chg_eof)
STps-eof = ST_NOEOF;
 
+   if (cmd_in == MTWEOF)
+   STps-rw = ST_IDLE;
} else { /* SCSI command was not completely successful. Don't return
 from this block without releasing the SCSI command block! 
*/
struct st_cmdstatus *cmdstatp = STp-buffer-cmdstat;
@@ -3235,6 +3237,17 @@ static int st_ioctl(struct inode *inode,
retval = i;
goto out;
}
+   if (STps-rw == ST_WRITING 
+   (mtc.mt_op == MTREW || mtc.mt_op == MTOFFL ||
+mtc.mt_op == MTSEEK)) {
+   i = st_int_ioctl(STp, MTWEOF, 1);
+   if (i  0) {
+   retval = i;
+   goto out;
+   }
+   STps-rw = ST_IDLE;
+}
+
} else {
/*
 * If there was a bus reset, block further access
--- linux-2.6.11-rc4-k1/Documentation/scsi/st.txt   2004-04-04 
15:34:34.0 +0300
+++ linux-2.6.11-rc4-k2/Documentation/scsi/st.txt   2005-02-20 
22:28:27.0 +0200
@@ -2,7 +2,7 @@ This file contains brief information abo
 The driver is currently maintained by Kai Mäkisara (email
 [EMAIL PROTECTED])
 
-Last modified: Wed Feb 25 14:09:08 2004 by makisara
+Last modified: Sun Feb 20 22:28:27 2005 by kai.makisara
 
 
 BASICS
@@ -85,6 +85,9 @@ writing and the last operation has been 
 optionally written. In both cases end of data is signified by
 returning zero bytes for two consecutive reads.
 
+If the rewind, offline, or seek is done and previous tape operation was write,
+a filemark is written before moving tape.
+
 The compile options are defined in the file linux/drivers/scsi/st_options.h.
 
 4. If the open option O_NONBLOCK is used, open succeeds even if the

[RFC,PATCH] Statistics for SCSI tapes

[Kai Makisara]

The patch at the end of this message adds a stat file to the primary st 
directory in /sys of each tape (e.g., /sys/class/scsi_tape/st0/stat). The 
contents of the file match the contents of the stat files in the 
subdirectories of /sys/block so that the same tools could be used to 
analyze all stat files (provided they find the stat files).

Some Unices implement statistics for each tape partition but this patch 
does not count the partitions separately.

The patch is against 2.6.11-rc4 + the descriptor based sense patch I sent 
on Feb 13 + the patch sent a few minutes ago. The patched driver compiles, 
did not crash the system and produced reasonable looking numbers.

Signed-off-by: Kai Makisara [EMAIL PROTECTED]

--- linux-2.6.11-rc4-k2/drivers/scsi/st.c   2005-02-20 22:22:51.0 
+0200
+++ linux-2.6.11-rc4-k3/drivers/scsi/st.c   2005-02-21 22:07:32.0 
+0200
@@ -392,6 +392,14 @@ static void st_sleep_done(struct scsi_cm
 {
struct scsi_tape *STp = 
container_of(SCpnt-request-rq_disk-private_data,
 struct scsi_tape, driver);
+   unsigned long ticks = jiffies - STp-tapestats.stamp;
+
+   STp-tapestats.io_ticks += ticks;
+   STp-tapestats.in_flight--;
+   if (SCpnt-cmnd[0] == WRITE_6)
+   STp-tapestats.write_ticks += ticks;
+   else if (SCpnt-cmnd[0] == READ_6)
+   STp-tapestats.read_ticks += ticks;
 
(STp-buffer)-cmdstat.midlevel_result = SCpnt-result;
SCpnt-request-rq_status = RQ_SCSI_DONE;
@@ -423,6 +431,16 @@ st_do_scsi(struct scsi_request * SRpnt, 
}
}
 
+   if (cmd[0] == WRITE_6) {
+   STp-tapestats.write_bytes += bytes;
+   STp-tapestats.writes++;
+   }
+   else if (cmd[0] == READ_6) {
+   STp-tapestats.read_bytes += bytes;
+   STp-tapestats.reads++;
+   }
+   STp-tapestats.in_flight++;
+
init_completion(STp-wait);
SRpnt-sr_use_sg = STp-buffer-do_dio || (bytes  
(STp-buffer)-frp[0].length);
if (SRpnt-sr_use_sg) {
@@ -439,6 +457,8 @@ st_do_scsi(struct scsi_request * SRpnt, 
SRpnt-sr_request-rq_disk = STp-disk;
STp-buffer-cmdstat.have_sense = 0;
 
+   STp-tapestats.stamp = jiffies;
+
scsi_do_req(SRpnt, (void *) cmd, bp, bytes,
st_sleep_done, timeout, retries);
 
@@ -3957,6 +3977,7 @@ static int st_probe(struct device *dev)
STm-cdevs[j] = cdev;
 
}
+   STm-tapestats = tpnt-tapestats;
do_create_class_files(tpnt, dev_num, mode);
}
 
@@ -4237,6 +4258,28 @@ static ssize_t st_defcompression_show(st
 
 CLASS_DEVICE_ATTR(default_compression, S_IRUGO, st_defcompression_show, NULL);
 
+static ssize_t st_stat_show(struct class_device *class_dev, char *buf)
+{
+   struct st_modedef *STm = (struct st_modedef 
*)class_get_devdata(class_dev);
+   ssize_t l = 0;
+
+   l = snprintf(buf, PAGE_SIZE,
+%8u %8u %8llu %8u 
+%8u %8u %8llu %8u 
+%8u %8u %8u
+\n,
+STm-tapestats-reads, 0, STm-tapestats-read_bytes  9,
+jiffies_to_msecs(STm-tapestats-read_ticks),
+STm-tapestats-writes, 0, STm-tapestats-write_bytes  
9,
+jiffies_to_msecs(STm-tapestats-write_ticks),
+STm-tapestats-in_flight,
+jiffies_to_msecs(STm-tapestats-io_ticks),
+jiffies_to_msecs(STm-tapestats-io_ticks));
+   return l;
+}
+
+CLASS_DEVICE_ATTR(stat, S_IRUGO, st_stat_show, NULL);
+
 static void do_create_class_files(struct scsi_tape *STp, int dev_num, int mode)
 {
int i, rew, error;
@@ -4273,6 +4316,8 @@ static void do_create_class_files(struct
class_device_create_file(st_class_member,
 
class_device_attr_default_compression);
if (mode == 0  rew == 0) {
+   class_device_create_file(st_class_member,
+class_device_attr_stat);
error = 
sysfs_create_link(STp-device-sdev_gendev.kobj,
  st_class_member-kobj,
  tape);
--- linux-2.6.11-rc4-k2/drivers/scsi/st.h   2005-02-13 10:40:21.0 
+0200
+++ linux-2.6.11-rc4-k3/drivers/scsi/st.h   2005-02-21 21:45:11.0 
+0200
@@ -57,6 +57,7 @@ struct st_modedef {
unsigned char default_compression;  /* 0 = don't touch, etc */
short default_density;  /* Forced density, -1 = no value */
int default_blksize;/* Forced blocksize, -1 = no value */
+   struct tape_stats *tapestats;
struct cdev *cdevs[2];  /* Auto-rewind and non-rewind devices */
 };
 
@@ -83,6 +84,16 @@ struct st_partstat {
 
 #define

Re: Fwd: Multi tape problems with cpio

[Kai Makisara]

On Tue, 18 Jan 2005, Tape Help wrote:

 Ok, I have the debug info, with comments where needed.
 Thanks alot!
 
...
 # find home -depth|cpio -o --format=newc --block-size=128 -F /dev/st0
 14:24:48 kernel: st0: Block limits 1 - 16777215 bytes.
 14:24:48 kernel: st0: Mode sense. Length 11, medium 0, WBS 10, BLL 8
 14:24:48 kernel: st0: Density 40, tape length: 0, drv buffer: 1
 14:24:48 kernel: st0: Block size: 0, buffer size: 32768 (1 blocks).
^
This shows that your drive is in variable block mode.

 14:24:48 kernel: st: Succeeded to enlarge buffer to 65536 bytes (segs
 1-9, 4096).
 16:32:28 kernel: st0: Error: 802, cmd: a 0 1 0 0 0 Len: 65536
 16:32:28 kernel: Info fld=0x0, EOM Current st09:00: sense key None
 16:32:28 kernel: Additional sense indicates End-of-partition/medium detected
 16:32:28 kernel: st0: Async write error (write) 7fff.

This is actually a 'magic error code' within st: the previous write did 
see the EOM early warning but all data was written. The code 7fff is 
later interpreted as succesful write and reported as such but the next 
write then returns the EOM error.

Next I would expect to see a message telling that one filemark is written. 
It can be done by the application but it is also automatically done when 
the device file is closed at this point (i.e., after write). But ...

 16:32:28 kernel: st0: Unloading tape.

at this point cpio ejects the tape and no filemark is written.

 16:32:58 kernel: st: Buffer at c031 normalized to 32768 bytes (segs 9).
 # cpio ejected the tape and was waiting for another.
 # I hit cntrlc
 # I put the tape back in
 # cpio -i --only-verify-crc --list --block-size=128 /dev/st0
 16:40:52 kernel: st0: Error: 802, cmd: 0 0 0 0 0 0 Len: 0
 16:40:52 kernel: Current st09:00: sense key Unit Attention
 16:40:52 kernel: Additional sense indicates Not ready to ready
 change,medium may have changed
 16:40:52 kernel: st0: Block limits 1 - 16777215 bytes.
 16:40:52 kernel: st0: Mode sense. Length 11, medium 0, WBS 10, BLL 8
 16:40:52 kernel: st0: Density 40, tape length: 0, drv buffer: 1
 16:40:52 kernel: st0: Block size: 0, buffer size: 32768 (1 blocks).
 16:40:52 kernel: st: Succeeded to enlarge buffer to 65536 bytes (segs
 1-9, 4096).
 18:45:58 kernel: st0: Error: 802, cmd: 8 0 1 0 0 0 Len: 65536
 18:45:58 kernel: Info fld=0x1, Current st09:00: sense key Blank Check
 18:45:58 kernel: Additional sense indicates End-of-data detected
 18:45:58 kernel: st0: Sense: f0  0  8  0  1  0  0  e

Now st encounters end of data and this is reported as read error.

My drive behaves in a slightly different way: it returns the same data but 
it also sets the EOM bit. In this case the st driver interprets the 
situation as normal end of data assuming that this is where the writing 
application stopped writing.

 18:45:58 kernel: st0: Tape error while reading.
 18:45:58 kernel: st0: Rewinding tape.
 18:46:07 kernel: st: Buffer at c031 normalized to 32768 bytes (segs 9).
 # cpio give this error: cpio: read error: Input/output error
 

So, your debugging data and my debugging data revealed what was happening 
and why we had different results. It is debatable what is the basic 
problem. The st driver is working as it has been designed to work but this does 
not match the expectations of cpio. My opinion is that any application 
should at least try to write a filemark at the end of each tape file and 
not rely on certain behaviour of drives and drivers at the end of an 
incomplete file. This is especially important because there is no common 
standard for tape semantics.

The problem can be solved by either changing st semantics or cpio 
behavior. Before changing st semantics I would like to be convinced that 
the changed semantics is what all (most) other unix tape drivers use. Any 
change of semantics can break other applications.

I will forward this analysis (with a preface) to the cpio maintainers.

-- 
Kai
-
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Zero Copy IO

[Kai Makisara]

This answer is longish and I send this only to linux-scsi to save
linux-kernel bandwidth.

On Sun, 8 Apr 2001, Alex Q Chen wrote:

 I am trying to find a way to pin down user space memory from kernel, so
 that these user space buffer can be used for direct IO transfer or
 otherwise known as "zero copying IO".  Searching through the Internet and
 reading comments on various news groups, it would appear that most
 developers including Linus himself doesn't believe in the benefit of "zero
 copying IO".  Most of the discussion however was based on network card
 drivers.  For certain other drivers such as SCSI Tape driver, which need to
 handle great deal of data transfer, it would seemed still be more
 advantageous to enable zero copy IO than copy_from_user() and copy_to_user
 () all the data.  Other OS such as AIX and OS2 have kernel functions that

Whether zero-copy for tapes is more efficient than the current method
using a kernel buffer depends on several factors and there is no unique
answer. If we want to maximize the usage of a CPU in a multitasking system
and don't care about the speed of the task writing to tape, zero-copy may
be better. This is assuming the transfers are long enough compared to the
increased overhead. (I would like to see someone to provide a quantitative
answer what is the minimum transfer size where zero-copy is advantageous).

On the other hand, if we want to maximize the throughput of the task
writing to tape, this gets more complicated. The memory-memory transfers
are much faster than the memory-tape buffer transfers. If we have some
spare bus cycles, the extra transfer is not a decisive factor.

The current tape driver uses by default so-called "asynchronous writes".
This means that a SCSI write is started and the write() function does not
wait for it to finish. The status is checked at the next write(). The user
task can use the SCSI transaction time to collect more data for writing.
This is not possible with zero-copy if we want to retain the Unix write
semantics. (Well, we are not completely obeying the semantics with
asynchronous writes because error reporting is delayed. However, this is
the case anyway if we enable drive buffering. Without drive buffering we
don't get decent throughput.)

In fixed block mode the driver can buffer several write()s worth of data
and this may increase throughput. The same applies to reading in fixed
block mode. Reading in variable block mode is a case where no speedup is
used.

If we use kernel space buffers, we can increase the parallelism without
changes to the user programs. How far this is useful is another question.
When using zero-copy, a program can use parallelism but this requires
changes to the program (multiple buffers and some kind of asynchrous i/o
framework).

Earlier it has been easy to postpone any discussion about using zero-copy
in the drivers. As you have seen from answers from others, this is
changing and fairly soon we will have the the necessary support for
zero-copy in the kernel. I am not against modifying the tape driver to use
zero-copy, but before doing it I would like to see/do a quantitative
analysis of the advantages and disadvantages in the common and not so
common cases.

Kai


-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to [EMAIL PROTECTED]