On 11/28/2015 6:10 PM, Matthew Grooms wrote:
On 11/27/2015 7:44 PM, Matthew Grooms wrote:
I spent the day looking over the FreeBSD cam and scsi_da source code.
After sprinkling a bunch of printf's around to see what code paths
were being called, It's obvious that Edward was correct in assuming
that ESXi doesn't return any 'Unit Attention' sense information in
response to a 'Read Capacity' request. This kinda makes sense as ESXi
emulates SCSI-2 disk devices and, as far as I can tell, the 0x2A/0x09
ASC/ASCQ sense code that denotes 'Capacity Data Has Changed' wasn't
defined until the SCSCI-3 spec. It's frustrating that the only way to
get the scsci_da code to call reprobe() is by receiving a command
from the device. Would something like this work? ...
1) Register a callback using xpt_register_async( daasync,
AC_REPROBE_DEVICE, path ) that calls reprobe()
2) Implement a new IOCTL in cam_xpt that camcontrol can call with the
bus:target:lun as the argument
3) have cam_xpt capture the IOCTL request and call xpt_async(
AC_REPROBE_DEVICE, path ) as a result
This way users would have the option of manually asking cam to
communicate the new size to geom. The only option now is one or more
reboots to gain access to the increased disk capacity. If this sounds
like a reasonable approach, I'll take a stab at implementing it.
Here is a proof of concept patch. I'm a complete noob when it comes to
cam, scsi or freebsd kernel development for that matter, so I'm sure
it could have been done a better way. In any case, I added a new
command to camcontrol that allows you to specify a bus, target and lun
as an argument. For example ...
# camcontrol readcap da1 -h
Device Size: 32 G, Block Length: 512 bytes
# gpart show da1
=> 40 58720176 da1 GPT (28G)
40 58720176 1 freebsd-ufs (28G)
Note, I resized the VMDK disk in ESXi. The camcontrol output shows the
size as 32G but geom thinks its 28G.
# camcontrol devlist
<NECVMWar VMware IDE CDR10 1.00> at scbus1 target 0 lun 0 (cd0,pass0)
<VMware Virtual disk 1.0> at scbus2 target 0 lun 0 (pass1,da0)
<VMware Virtual disk 1.0> at scbus2 target 1 lun 0 (pass2,da1)
<FREEBSD CTLDISK 0001> at scbus3 target 0 lun 0 (da2,pass3)
# camcontrol reprobe 2:1:0
This generates an event that is captured by the scsci da device to
forces a reprobe. The kernel output looks almost identical to when the
'Unit Attention' sense data is received ...
Nov 28 17:46:13 iscsi-i kernel: (da1:mpt0:0:1:0): Re-probe requested
Nov 28 17:46:13 iscsi-i kernel: GEOM_PART: da1 was automatically resized.
Nov 28 17:46:13 iscsi-i kernel: Use `gpart commit da1` to save changes
or `gpart undo da1` to revert them.
Now that geom knows about the increased disk capacity, I can increase
the partition size and grow the fs ...
[root@iscsi-i /home/mgrooms]# gpart show da1
=> 40 67108784 da1 GPT (32G)
40 58720176 1 freebsd-ufs (28G)
58720216 8388608 - free - (4.0G)
# gpart resize -i 1 da1
da1p1 resized
# growfs da1p1
Device is mounted read-write; resizing will result in temporary write
suspension for /var/data1.
It's strongly recommended to make a backup before growing the file
system.
OK to grow filesystem on /dev/da1p1, mounted on /var/data1, from 28GB
to 32GB? [Yes/No] Yes
super-block backups (for fsck_ffs -b #) at:
58983232, 60265472, 61547712, 62829952, 64112192, 65394432, 66676672
# df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/da0p3 18G 5.3G 12G 31% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/da1p1 31G 32M 28G 0% /var/data1
/dev/da2p1 15G 32M 14G 0% /var/data2
Sure would be nice to have something like this in the tree. It's
really a drag to have to reboot production VMs to increase disk
capacity when it could be easily avoided. I'm not sure what the
correct IOCTL should look like. Maybe CAMIOCOMMAND is a better way to
go? If someone with some experience with the cam/scsi subsystems was
willing to give me some direction I'd be willing to try and rewrite
the patch in a way that would be commit worthy. I just need some
direction.
Ok, last post until I get some feedback. Here's a new version of the
patch complete with man page updates. It communicates via CAMIOCOMMAND
instead of introducing a new ioctl value. I tried to model it after the
device reset option, hopefully with some degree of success. Functionally
it should be the same as the first patch.
Thanks,
-Matthew
Index: sbin/camcontrol/camcontrol.8
===================================================================
--- sbin/camcontrol/camcontrol.8 (revision 291390)
+++ sbin/camcontrol/camcontrol.8 (working copy)
@@ -104,6 +104,9 @@
.Ic reset
.Aq all | bus Ns Op :target:lun
.Nm
+.Ic reprobe
+.Aq bus:target:lun
+.Nm
.Ic defects
.Op device id
.Op generic args
@@ -548,6 +551,9 @@
connecting to that device.
Note that this can have a destructive impact
on the system.
+.It Ic reprobe
+Tell the kernel to re-probe the given bus:target:lun
+(XPT_REPROBE_DEV) by issuing an async event to peripheral device.
.It Ic defects
Send the
.Tn SCSI
Index: sbin/camcontrol/camcontrol.c
===================================================================
--- sbin/camcontrol/camcontrol.c (revision 291390)
+++ sbin/camcontrol/camcontrol.c (working copy)
@@ -100,7 +100,8 @@
CAM_CMD_APM = 0x00000021,
CAM_CMD_AAM = 0x00000022,
CAM_CMD_ATTRIB = 0x00000023,
- CAM_CMD_OPCODES = 0x00000024
+ CAM_CMD_OPCODES = 0x00000024,
+ CAM_CMD_REPROBE = 0x00000025
} cam_cmdmask;
typedef enum {
@@ -193,6 +194,7 @@
#endif /* MINIMALISTIC */
{"rescan", CAM_CMD_RESCAN, CAM_ARG_NONE, NULL},
{"reset", CAM_CMD_RESET, CAM_ARG_NONE, NULL},
+ {"reprobe", CAM_CMD_REPROBE, CAM_ARG_NONE, NULL},
#ifndef MINIMALISTIC
{"cmd", CAM_CMD_SCSI_CMD, CAM_ARG_NONE, scsicmd_opts},
{"command", CAM_CMD_SCSI_CMD, CAM_ARG_NONE, scsicmd_opts},
@@ -273,6 +275,7 @@
static int rescan_or_reset_bus(path_id_t bus, int rescan);
static int scanlun_or_reset_dev(path_id_t bus, target_id_t target,
lun_id_t lun, int scan);
+static int reprobe_dev(path_id_t bus, target_id_t target, lun_id_t lun);
#ifndef MINIMALISTIC
static int readdefects(struct cam_device *device, int argc, char **argv,
char *combinedopt, int retry_count, int timeout);
@@ -3127,6 +3130,34 @@
}
static int
+doreprobe(int argc, char **argv)
+{
+ static const char must[] =
+ "you must specify a bus:target:lun to reprobe";
+ int rv;
+ path_id_t bus = CAM_BUS_WILDCARD;
+ target_id_t target = CAM_TARGET_WILDCARD;
+ lun_id_t lun = CAM_LUN_WILDCARD;
+ char *tstr;
+
+ if (argc < 3) {
+ warnx(must);
+ return(1);
+ }
+
+ tstr = argv[optind];
+ while (isspace(*tstr) && (*tstr != '\0'))
+ tstr++;
+ rv = parse_btl(argv[optind], &bus, &target, &lun, &arglist);
+ if (rv != 3) {
+ warnx(must);
+ return(1);
+ }
+
+ return(reprobe_dev(bus, target, lun));
+}
+
+static int
rescan_or_reset_bus(path_id_t bus, int rescan)
{
union ccb ccb, matchccb;
@@ -3382,6 +3413,59 @@
}
}
+static int
+reprobe_dev(path_id_t bus, target_id_t target, lun_id_t lun)
+{
+ union ccb ccb;
+ struct cam_device *device;
+ int fd;
+
+ device = NULL;
+
+ if (bus == CAM_BUS_WILDCARD) {
+ warnx("invalid bus number %d", bus);
+ return(1);
+ }
+
+ if (target == CAM_TARGET_WILDCARD) {
+ warnx("invalid target number %d", target);
+ return(1);
+ }
+
+ if (lun == CAM_LUN_WILDCARD) {
+ warnx("invalid lun number %jx", (uintmax_t)lun);
+ return(1);
+ }
+
+ fd = -1;
+
+ bzero(&ccb, sizeof(union ccb));
+
+ if ((fd = open(XPT_DEVICE, O_RDWR)) < 0) {
+ warnx("error opening transport layer device %s\n",
+ XPT_DEVICE);
+ warn("%s", XPT_DEVICE);
+ return(1);
+ }
+
+ ccb.ccb_h.func_code = XPT_REPROBE_DEV;
+ ccb.ccb_h.path_id = bus;
+ ccb.ccb_h.target_id = target;
+ ccb.ccb_h.target_lun = lun;
+
+ if (ioctl(fd, CAMIOCOMMAND, &ccb) < 0) {
+ warn("CAMIOCOMMAND ioctl failed");
+ close(fd);
+ return(1);
+ }
+ close(fd);
+
+ fprintf(stdout, "Re-probe of %d:%d:%jx was successful\n",
+ bus, target, (uintmax_t)lun);
+
+ return(0);
+}
+
#ifndef MINIMALISTIC
static struct scsi_nv defect_list_type_map[] = {
@@ -8685,6 +8769,7 @@
#endif /* MINIMALISTIC */
" camcontrol rescan <all | bus[:target:lun]>\n"
" camcontrol reset <all | bus[:target:lun]>\n"
+" camcontrol reprobe bus:target:lun\n"
#ifndef MINIMALISTIC
" camcontrol defects [dev_id][generic args] <-f format> [-P][-G]\n"
" [-q][-s][-S offset][-X]\n"
@@ -9053,6 +9138,7 @@
*/
if ((cmdlist == CAM_CMD_RESCAN)
|| (cmdlist == CAM_CMD_RESET)
+ || (cmdlist == CAM_CMD_REPROBE)
|| (cmdlist == CAM_CMD_DEVTREE)
|| (cmdlist == CAM_CMD_USAGE)
|| (cmdlist == CAM_CMD_DEBUG))
@@ -9204,6 +9290,9 @@
case CAM_CMD_RESET:
error = dorescan_or_reset(argc, argv, 0);
break;
+ case CAM_CMD_REPROBE:
+ error = doreprobe(argc, argv);
+ break;
#ifndef MINIMALISTIC
case CAM_CMD_READ_DEFECTS:
error = readdefects(cam_dev, argc, argv, combinedopt,
Index: sys/amd64/conf/GENERIC
===================================================================
--- sys/amd64/conf/GENERIC (revision 291390)
+++ sys/amd64/conf/GENERIC (working copy)
@@ -366,3 +366,6 @@
# The crypto framework is required by IPSEC
device crypto # Required by IPSEC
+
+# Cam Debug Support
+options CAMDEBUG
Index: sys/cam/cam_ccb.h
===================================================================
--- sys/cam/cam_ccb.h (revision 291390)
+++ sys/cam/cam_ccb.h (working copy)
@@ -193,6 +193,11 @@
* Set SIM specific knob values.
*/
+ XPT_REPROBE_DEV = 0x1a,
+ /*
+ * Re-probe the sepcified SCSI device.
+ */
+
XPT_SMP_IO = 0x1b | XPT_FC_DEV_QUEUED,
/* Serial Management Protocol */
@@ -767,6 +772,7 @@
* Definitions for the asynchronous callback CCB fields.
*/
typedef enum {
+ AC_UNIT_REPROBE = 0x8000,/* Device reprobe user request */
AC_UNIT_ATTENTION = 0x4000,/* Device reported UNIT ATTENTION */
AC_ADVINFO_CHANGED = 0x2000,/* Advance info might have changes */
AC_CONTRACT = 0x1000,/* A contractual callback */
Index: sys/cam/cam_xpt.c
===================================================================
--- sys/cam/cam_xpt.c (revision 291390)
+++ sys/cam/cam_xpt.c (working copy)
@@ -433,6 +433,13 @@
return (EINVAL);
}
break;
+ case XPT_REPROBE_DEV:
+ if (inccb->ccb_h.target_id == CAM_TARGET_WILDCARD ||
+ inccb->ccb_h.target_lun == CAM_LUN_WILDCARD) {
+ xpt_release_bus(bus);
+ return (EINVAL);
+ }
+ break;
default:
break;
}
@@ -472,6 +479,29 @@
xpt_free_ccb(ccb);
break;
+ case XPT_REPROBE_DEV: {
+ struct cam_path path;
+
+ /*
+ * Compile a path using the bus, target, and lun the
+ * user passed in.
+ */
+ if (xpt_compile_path(&path, NULL,
+ inccb->ccb_h.path_id,
+ inccb->ccb_h.target_id,
+ inccb->ccb_h.target_lun) !=
+ CAM_REQ_CMP){
+ error = EINVAL;
+ break;
+ }
+
+ /*
+ * Notify interested parties
+ */
+ xpt_async(AC_UNIT_REPROBE, &path, NULL);
+ break;
+
+ }
case XPT_DEBUG: {
union ccb ccb;
@@ -717,7 +747,7 @@
}
xpt_unlock_buses();
break;
- }
+ }
default:
error = ENOTTY;
break;
Index: sys/cam/scsi/scsi_da.c
===================================================================
--- sys/cam/scsi/scsi_da.c (revision 291390)
+++ sys/cam/scsi/scsi_da.c (working copy)
@@ -1702,6 +1702,17 @@
}
break;
}
+ case AC_UNIT_REPROBE:
+ {
+ softc = (struct da_softc *)periph->softc;
+
+ xpt_print(periph->path,
+ "Re-probe requested\n");
+ softc->flags &= ~DA_FLAG_PROBED;
+ dareprobe(periph);
+
+ break;
+ }
case AC_UNIT_ATTENTION:
{
union ccb *ccb;
@@ -2240,7 +2251,7 @@
* would be to not attach the device on failure.
*/
xpt_register_async(AC_SENT_BDR | AC_BUS_RESET | AC_LOST_DEVICE |
- AC_ADVINFO_CHANGED | AC_SCSI_AEN | AC_UNIT_ATTENTION,
+ AC_ADVINFO_CHANGED | AC_SCSI_AEN | AC_UNIT_ATTENTION |
AC_UNIT_REPROBE,
daasync, periph, periph->path);
/*
@@ -3244,12 +3255,11 @@
if (have_sense)
scsi_sense_print(
&done_ccb->csio);
- else {
+ else
xpt_print(periph->path,
"got CAM status %#x\n",
done_ccb->ccb_h.status);
- }
-
+
xpt_print(periph->path, "fatal error, "
"failed to attach to device\n");
_______________________________________________
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
