Re: zfs, iSCSI and volmode=dev

[Edward Napierala]

2017-10-09 17:38 GMT+01:00 Eugene M. Zheganin :

> Hi,
>
> On 27.09.2017 16:07, Edward Napierala wrote:
>
>> 2017-08-30 11:45 GMT+02:00 Eugene M. Zheganin > e...@norma.perm.ru>>:
>>
>> Hi,
>>
>>
>> I have an iSCSI production system that exports a large number of
>> zvols as the iSCSI targets. System is running FreeBSD
>> 11.0-RELEASE-p7 and initially all of the zvols were confugured
>> with default volmode. I've read that it's recommended to use them
>> in dev mode, so the system isn't bothered with all of these geom
>> structures, so I've switched all of the zvols to dev mode, then I
>> exported/imported the pools back. Surprisingly, the performance
>> has fallen down like 10 times (200-300 Mbits/sec against 3-4
>> Gbits/sec previously). After observing for 5 minutes the ESXes
>> trying to boot up, and doing this extremely slowly, I switched the
>> volmode back to default, then again exported/imported the pools.
>> The performance went back to normal.
>>
>>
>> So... why did this happen ? The result seems to be
>> counter-intuitive. At least not obvious to me.
>>
>>
>> I don't really have an answer - mav@ would be the best person to ask.
>> Based
>> on his description, "ZVOLs in GEOM mode don't support DPO/FUA cache
>> control
>> bits, had to chunk large I/Os into MAXPHYS-sized pieces and go through
>> GEOM."
>> There also used to be so that TRIM was only supported in the "dev" mode,
>> but
>> that changed a while ago.
>>
>> Yeah, but you mean dev is faster by design. So was my first thought too,
> but it seems like the opposite. Default volmode is geom, and it's much
> faster than dev.


I'd expect it to be faster, but it might be it interferes with something.
For example,
if Windows forces direct media access (bypassing the disk cache), and going
through
GEOM made the target ignore this bit, that could hurt performance.  (Note
that
it's just an idea; I have no idea what's actually happening there.)

My first suspect would be TRIM, but then it should work in both modes.
Might be worth
checking to be sure, by disabling the "unmap" option in ctl.conf (option
"unmap" "off")
to see if it makes a difference.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: zfs, iSCSI and volmode=dev

[Eugene M. Zheganin]

Hi,

On 27.09.2017 16:07, Edward Napierala wrote:
2017-08-30 11:45 GMT+02:00 Eugene M. Zheganin >:


Hi,


I have an iSCSI production system that exports a large number of
zvols as the iSCSI targets. System is running FreeBSD
11.0-RELEASE-p7 and initially all of the zvols were confugured
with default volmode. I've read that it's recommended to use them
in dev mode, so the system isn't bothered with all of these geom
structures, so I've switched all of the zvols to dev mode, then I
exported/imported the pools back. Surprisingly, the performance
has fallen down like 10 times (200-300 Mbits/sec against 3-4
Gbits/sec previously). After observing for 5 minutes the ESXes
trying to boot up, and doing this extremely slowly, I switched the
volmode back to default, then again exported/imported the pools.
The performance went back to normal.


So... why did this happen ? The result seems to be
counter-intuitive. At least not obvious to me.


I don't really have an answer - mav@ would be the best person to ask.  
Based
on his description, "ZVOLs in GEOM mode don't support DPO/FUA cache 
control
bits, had to chunk large I/Os into MAXPHYS-sized pieces and go through 
GEOM."
There also used to be so that TRIM was only supported in the "dev" 
mode, but

that changed a while ago.

Yeah, but you mean dev is faster by design. So was my first thought too, 
but it seems like the opposite. Default volmode is geom, and it's much 
faster than dev.


Eugene.


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: zfs, iSCSI and volmode=dev

[Edward Napierala]

2017-08-30 11:45 GMT+02:00 Eugene M. Zheganin :

> Hi,
>
>
> I have an iSCSI production system that exports a large number of zvols as
> the iSCSI targets. System is running FreeBSD 11.0-RELEASE-p7 and initially
> all of the zvols were confugured with default volmode. I've read that it's
> recommended to use them in dev mode, so the system isn't bothered with all
> of these geom structures, so I've switched all of the zvols to dev mode,
> then I exported/imported the pools back. Surprisingly, the performance has
> fallen down like 10 times (200-300 Mbits/sec against 3-4 Gbits/sec
> previously). After observing for 5 minutes the ESXes trying to boot up, and
> doing this extremely slowly, I switched the volmode back to default, then
> again exported/imported the pools. The performance went back to normal.
>
>
> So... why did this happen ? The result seems to be counter-intuitive. At
> least not obvious to me.


I don't really have an answer - mav@ would be the best person to ask.  Based
on his description, "ZVOLs in GEOM mode don't support DPO/FUA cache control
bits, had to chunk large I/Os into MAXPHYS-sized pieces and go through
GEOM."
There also used to be so that TRIM was only supported in the "dev" mode, but
that changed a while ago.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: zfs, iSCSI and volmode=dev

[Pete French]

I have an iSCSI production system that exports a large number of zvols 
as the iSCSI targets. System is running FreeBSD 11.0-RELEASE-p7 and 
initially all of the zvols were confugured with default volmode. I've 
read that it's recommended to use them in dev mode, so the system isn't 
bothered with all of these geom structures, so I've switched all of the 
zvols to dev mode, then I exported/imported the pools back. 
Surprisingly, the performance has fallen down like 10 times (200-300 
Mbits/sec against 3-4 Gbits/sec previously). After observing for 5 
minutes the ESXes trying to boot up, and doing this extremely slowly, I 
switched the volmode back to default, then again exported/imported the 
pools. The performance went back to normal.


I too ghave a number of zvols exportes as iscsi, and I was using dev 
mode util I read this. AM going to experiment with switching back to 
default mode to see what happens. I also use dev mode for my virtualbox 
drives. Am going to try and test that too and see what I get.


Interesting result though...

-pete.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"