Re: MS DNS doesn't answer to CURRENT under Hyper-V

2015-07-29 Thread Pavel Timofeev
Hi!
r285785 still isn't MFCed.
RC2 is coming soon.

2015-07-23 10:54 GMT+03:00 Pavel Timofeev tim...@gmail.com:
 Ok, sorry!

 2015-07-23 7:51 GMT+03:00 Wei Hu w...@microsoft.com:
 The TCP offloading is still working on these platforms. There is no flag to 
 distinguish UDP and TCP offloading, so the RXCSUM and TXCSUM are still set. 
 Let me know if there is any other way to show it properly.

 Thanks,
 Wei


 -Original Message-
 From: Pavel Timofeev [mailto:tim...@gmail.com]
 Sent: Wednesday, July 22, 2015 9:04 PM
 To: Wei Hu w...@microsoft.com
 Cc: Slawa Olhovchenkov s...@zxy.spb.ru; freebsd-curr...@freebsd.org; 
 freebsd-virtualization@freebsd.org
 Subject: Re: MS DNS doesn't answer to CURRENT under Hyper-V

 Hi! I see you have done the code for disabling UDP checksum offloading when 
 running on the Hyper-V on Windows Server 2012 and earlier hosts

 https://svnweb.freebsd.org/base?view=revisionrevision=285785

 I tried new CURRENT and it works. Thank you!

 A small note here: while it disables and works it still shows RXCSUM and 
 TSCSUM in iface's options:

 root@proxy:/usr/src # ifconfig hn0
 hn0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
 options=31bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,TSO6
 ether 00:15:5d:02:9c:09
 inet 192.168.25.26 netmask 0xffc0 broadcast 192.168.25.63
 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL

 Is it possible to hide it automatically if it's disabled by new code?


 2015-07-13 11:06 GMT+03:00 Wei Hu w...@microsoft.com:
 We have root caused the problem. This issue happens on the Hyper-Vs on 
 Windows Server 2012 (Win 8.0) and earlier releases. On these releases, the 
 UPD checksum offloading on host side does not work properly. The workaround 
 is to disable UPD checksum offloading in the FreeBSD guest through 
 'ifconfig'. We are also working on a patch to turn off UPD checksum 
 offloading in the netvsc driver when detecting the Hyper-V releases.

 The UDP checksum offloading works fine on Windows Server 2012R2 and Win 8.1 
 hosts.

 Thanks Pavel and Slawa for the support.

 Wei


 -Original Message-
 From: owner-freebsd-virtualizat...@freebsd.org [mailto:owner-freebsd-
 virtualizat...@freebsd.org] On Behalf Of Pavel Timofeev
 Sent: Wednesday, July 8, 2015 4:06 PM
 To: Slawa Olhovchenkov
 Cc: freebsd-curr...@freebsd.org; freebsd-virtualization@freebsd.org
 Subject: Re: MS DNS doesn't answer to CURRENT under Hyper-V

 Ok, r284746 is the root of the problem. MS DNS works under r284745
 and doesn't work under r284746.
 Slawa, what should I look at in wireshark output?


 2015-07-07 18:49 GMT+03:00 Slawa Olhovchenkov s...@zxy.spb.ru:
  On Tue, Jul 07, 2015 at 06:04:46PM +0300, Pavel Timofeev wrote:
 
  Well, turning off checksum offloading by `ifconfig hn0 -txcsum
  -rxcsum` definitely helps.
 
  As for tcpdump I'm not completely sure if I did it right, but I
  see bad udp cksum phrase:
 
  # tcpdump -i hn0 -vvv -nn udp dst port 53
  tcpdump: listening on hn0, link-type EN10MB (Ethernet), capture
  size
  262144 bytes
  18:01:19.139994 IP (tos 0x0, ttl 64, id 61218, offset 0, flags
  [none], proto UDP (17), length 51)
  192.168.25.26.45683  192.168.25.3.53: [bad udp cksum 0xb39e
  - 0xf210!] 52886+ A? ya.ru. (23)
  18:01:24.140544 IP (tos 0x0, ttl 64, id 17293, offset 0, flags
  [none], proto UDP (17), length 51)
  192.168.25.26.12575  192.168.25.3.53: [bad udp cksum 0xb39e
  - 0x7365!] 52886+ A? ya.ru. (23)
 
  tcpdump bad udp cksum is normal on FreeBSD host in case checksum
  offload (and may be need only for help finding issuse in code).
  Need wireshark capturing from MS DNS host (or from mirroring port).
 ___
 freebsd-virtualization@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
 To unsubscribe, send any mail to freebsd-virtualization-
 unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: MS DNS doesn't answer to CURRENT under Hyper-V

2015-07-23 Thread Pavel Timofeev
Ok, sorry!

2015-07-23 7:51 GMT+03:00 Wei Hu w...@microsoft.com:
 The TCP offloading is still working on these platforms. There is no flag to 
 distinguish UDP and TCP offloading, so the RXCSUM and TXCSUM are still set. 
 Let me know if there is any other way to show it properly.

 Thanks,
 Wei


 -Original Message-
 From: Pavel Timofeev [mailto:tim...@gmail.com]
 Sent: Wednesday, July 22, 2015 9:04 PM
 To: Wei Hu w...@microsoft.com
 Cc: Slawa Olhovchenkov s...@zxy.spb.ru; freebsd-curr...@freebsd.org; 
 freebsd-virtualization@freebsd.org
 Subject: Re: MS DNS doesn't answer to CURRENT under Hyper-V

 Hi! I see you have done the code for disabling UDP checksum offloading when 
 running on the Hyper-V on Windows Server 2012 and earlier hosts

 https://svnweb.freebsd.org/base?view=revisionrevision=285785

 I tried new CURRENT and it works. Thank you!

 A small note here: while it disables and works it still shows RXCSUM and 
 TSCSUM in iface's options:

 root@proxy:/usr/src # ifconfig hn0
 hn0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
 options=31bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,TSO6
 ether 00:15:5d:02:9c:09
 inet 192.168.25.26 netmask 0xffc0 broadcast 192.168.25.63
 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL

 Is it possible to hide it automatically if it's disabled by new code?


 2015-07-13 11:06 GMT+03:00 Wei Hu w...@microsoft.com:
 We have root caused the problem. This issue happens on the Hyper-Vs on 
 Windows Server 2012 (Win 8.0) and earlier releases. On these releases, the 
 UPD checksum offloading on host side does not work properly. The workaround 
 is to disable UPD checksum offloading in the FreeBSD guest through 
 'ifconfig'. We are also working on a patch to turn off UPD checksum 
 offloading in the netvsc driver when detecting the Hyper-V releases.

 The UDP checksum offloading works fine on Windows Server 2012R2 and Win 8.1 
 hosts.

 Thanks Pavel and Slawa for the support.

 Wei


 -Original Message-
 From: owner-freebsd-virtualizat...@freebsd.org [mailto:owner-freebsd-
 virtualizat...@freebsd.org] On Behalf Of Pavel Timofeev
 Sent: Wednesday, July 8, 2015 4:06 PM
 To: Slawa Olhovchenkov
 Cc: freebsd-curr...@freebsd.org; freebsd-virtualization@freebsd.org
 Subject: Re: MS DNS doesn't answer to CURRENT under Hyper-V

 Ok, r284746 is the root of the problem. MS DNS works under r284745
 and doesn't work under r284746.
 Slawa, what should I look at in wireshark output?


 2015-07-07 18:49 GMT+03:00 Slawa Olhovchenkov s...@zxy.spb.ru:
  On Tue, Jul 07, 2015 at 06:04:46PM +0300, Pavel Timofeev wrote:
 
  Well, turning off checksum offloading by `ifconfig hn0 -txcsum
  -rxcsum` definitely helps.
 
  As for tcpdump I'm not completely sure if I did it right, but I
  see bad udp cksum phrase:
 
  # tcpdump -i hn0 -vvv -nn udp dst port 53
  tcpdump: listening on hn0, link-type EN10MB (Ethernet), capture
  size
  262144 bytes
  18:01:19.139994 IP (tos 0x0, ttl 64, id 61218, offset 0, flags
  [none], proto UDP (17), length 51)
  192.168.25.26.45683  192.168.25.3.53: [bad udp cksum 0xb39e
  - 0xf210!] 52886+ A? ya.ru. (23)
  18:01:24.140544 IP (tos 0x0, ttl 64, id 17293, offset 0, flags
  [none], proto UDP (17), length 51)
  192.168.25.26.12575  192.168.25.3.53: [bad udp cksum 0xb39e
  - 0x7365!] 52886+ A? ya.ru. (23)
 
  tcpdump bad udp cksum is normal on FreeBSD host in case checksum
  offload (and may be need only for help finding issuse in code).
  Need wireshark capturing from MS DNS host (or from mirroring port).
 ___
 freebsd-virtualization@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
 To unsubscribe, send any mail to freebsd-virtualization-
 unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: MS DNS doesn't answer to CURRENT under Hyper-V

2015-07-22 Thread Pavel Timofeev
Hi! I see you have done the code for disabling UDP checksum offloading
when running on the Hyper-V on Windows Server 2012 and earlier hosts

https://svnweb.freebsd.org/base?view=revisionrevision=285785

I tried new CURRENT and it works. Thank you!

A small note here: while it disables and works it still shows RXCSUM
and TSCSUM in iface's options:

root@proxy:/usr/src # ifconfig hn0
hn0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST metric 0 mtu 1500
options=31bRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,TSO4,TSO6
ether 00:15:5d:02:9c:09
inet 192.168.25.26 netmask 0xffc0 broadcast 192.168.25.63
nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL

Is it possible to hide it automatically if it's disabled by new code?


2015-07-13 11:06 GMT+03:00 Wei Hu w...@microsoft.com:
 We have root caused the problem. This issue happens on the Hyper-Vs on 
 Windows Server 2012 (Win 8.0) and earlier releases. On these releases, the 
 UPD checksum offloading on host side does not work properly. The workaround 
 is to disable UPD checksum offloading in the FreeBSD guest through 
 'ifconfig'. We are also working on a patch to turn off UPD checksum 
 offloading in the netvsc driver when detecting the Hyper-V releases.

 The UDP checksum offloading works fine on Windows Server 2012R2 and Win 8.1 
 hosts.

 Thanks Pavel and Slawa for the support.

 Wei


 -Original Message-
 From: owner-freebsd-virtualizat...@freebsd.org [mailto:owner-freebsd-
 virtualizat...@freebsd.org] On Behalf Of Pavel Timofeev
 Sent: Wednesday, July 8, 2015 4:06 PM
 To: Slawa Olhovchenkov
 Cc: freebsd-curr...@freebsd.org; freebsd-virtualization@freebsd.org
 Subject: Re: MS DNS doesn't answer to CURRENT under Hyper-V

 Ok, r284746 is the root of the problem. MS DNS works under r284745 and
 doesn't work under r284746.
 Slawa, what should I look at in wireshark output?


 2015-07-07 18:49 GMT+03:00 Slawa Olhovchenkov s...@zxy.spb.ru:
  On Tue, Jul 07, 2015 at 06:04:46PM +0300, Pavel Timofeev wrote:
 
  Well, turning off checksum offloading by `ifconfig hn0 -txcsum
  -rxcsum` definitely helps.
 
  As for tcpdump I'm not completely sure if I did it right, but I see
  bad udp cksum phrase:
 
  # tcpdump -i hn0 -vvv -nn udp dst port 53
  tcpdump: listening on hn0, link-type EN10MB (Ethernet), capture size
  262144 bytes
  18:01:19.139994 IP (tos 0x0, ttl 64, id 61218, offset 0, flags
  [none], proto UDP (17), length 51)
  192.168.25.26.45683  192.168.25.3.53: [bad udp cksum 0xb39e -
  0xf210!] 52886+ A? ya.ru. (23)
  18:01:24.140544 IP (tos 0x0, ttl 64, id 17293, offset 0, flags
  [none], proto UDP (17), length 51)
  192.168.25.26.12575  192.168.25.3.53: [bad udp cksum 0xb39e -
  0x7365!] 52886+ A? ya.ru. (23)
 
  tcpdump bad udp cksum is normal on FreeBSD host in case checksum
  offload (and may be need only for help finding issuse in code). Need
  wireshark capturing from MS DNS host (or from mirroring port).
 ___
 freebsd-virtualization@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
 To unsubscribe, send any mail to freebsd-virtualization-
 unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: MS DNS doesn't answer to CURRENT under Hyper-V

2015-07-08 Thread Pavel Timofeev
Ok, r284746 is the root of the problem. MS DNS works under r284745 and
doesn't work under r284746.
Slawa, what should I look at in wireshark output?


2015-07-07 18:49 GMT+03:00 Slawa Olhovchenkov s...@zxy.spb.ru:
 On Tue, Jul 07, 2015 at 06:04:46PM +0300, Pavel Timofeev wrote:

 Well, turning off checksum offloading by `ifconfig hn0 -txcsum
 -rxcsum` definitely helps.

 As for tcpdump I'm not completely sure if I did it right, but I see
 bad udp cksum phrase:

 # tcpdump -i hn0 -vvv -nn udp dst port 53
 tcpdump: listening on hn0, link-type EN10MB (Ethernet), capture size
 262144 bytes
 18:01:19.139994 IP (tos 0x0, ttl 64, id 61218, offset 0, flags [none],
 proto UDP (17), length 51)
 192.168.25.26.45683  192.168.25.3.53: [bad udp cksum 0xb39e -
 0xf210!] 52886+ A? ya.ru. (23)
 18:01:24.140544 IP (tos 0x0, ttl 64, id 17293, offset 0, flags [none],
 proto UDP (17), length 51)
 192.168.25.26.12575  192.168.25.3.53: [bad udp cksum 0xb39e -
 0x7365!] 52886+ A? ya.ru. (23)

 tcpdump bad udp cksum is normal on FreeBSD host in case checksum
 offload (and may be need only for help finding issuse in code). Need
 wireshark capturing from MS DNS host (or from mirroring port).
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: MS DNS doesn't answer to CURRENT under Hyper-V

2015-07-07 Thread Pavel Timofeev
No, I don't even know what VMDepot is, sorry! Didn't know ;)

2015-07-07 16:58 GMT+03:00 Glen Barber g...@freebsd.org:
 BTW, are you using the images from VMDepot?  If so, this was MFC'd after
 the recent images.  I can easily regenerate an updated image, if you are
 using those.

 Glen

 On Tue, Jul 07, 2015 at 04:55:39PM +0300, Pavel Timofeev wrote:
 Wow, r284746 was MFCed to 10 STABLE!
 I should hurry up!

 2015-07-07 16:26 GMT+03:00 Pavel Timofeev tim...@gmail.com:
  Ok, I'll try r284745 and then r284746 and see what happens.
 
  2015-07-07 16:06 GMT+03:00 Wei Hu w...@microsoft.com:
  -Original Message-
  From: owner-freebsd-virtualizat...@freebsd.org [mailto:owner-freebsd-
  virtualizat...@freebsd.org] On Behalf Of Pavel Timofeev
  Sent: Tuesday, July 7, 2015 7:51 PM
  To: freebsd-curr...@freebsd.org; freebsd-virtualization@freebsd.org
  Subject: MS DNS doesn't answer to CURRENT under Hyper-V
 
  Hi!
  I have a test virtual machine which runs CURRENT under Hyper-V. It's
  amd64 r285198 now.
  It can't get any response from MS DNS server. Well, it could two or three
  weeks ago, but after upgrade it's not able to do it anymore.
  Google DNS answers without problems meanwhile (sic!).
 
  What I do:
  # host google.ru 192.168.25.3
  I see that MS DNS (192.168.25.3) server receives these packets, but 
  ignores
  them.
  And no matter how my system asks MS DNS. Every daemon can't get
  response too.
 
  I know that nothing was changed in MS DNS server. No doubt.
  Then I tried different available CURRENT snapshot ISOs.
 
  FreeBSD-11.0-CURRENT-amd64-20150630-r284969-disc1.iso - MS DNS does
  not answer.
 
  FreeBSD-11.0-CURRENT-amd64-20150625-r284814-disc1.iso - MS DNS does
  not answer.
 
  FreeBSD-11.0-CURRENT-amd64-20150618-r284544-disc1.iso - MS DNS
  answers!
 
  So something was committed to CURRENT between 20150618 and 20150625.
  This something ruins communication with MS DNS.
 
  There was a commit for Hyper-V TSO and checksum offloading support  
  (r284746) on
  June 24th. I think this commit is the cause. Can you verify the MS DNS 
  behavior between
  The builds of June 23rd and 24th? I will take a look of this issue 
  tomorrow.
 
  Thanks,
  Wei
 
 
 
 ___
 freebsd-virtualization@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
 To unsubscribe, send any mail to 
 freebsd-virtualization-unsubscr...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: MS DNS doesn't answer to CURRENT under Hyper-V

2015-07-07 Thread Pavel Timofeev
Well, turning off checksum offloading by `ifconfig hn0 -txcsum
-rxcsum` definitely helps.

As for tcpdump I'm not completely sure if I did it right, but I see
bad udp cksum phrase:

# tcpdump -i hn0 -vvv -nn udp dst port 53
tcpdump: listening on hn0, link-type EN10MB (Ethernet), capture size
262144 bytes
18:01:19.139994 IP (tos 0x0, ttl 64, id 61218, offset 0, flags [none],
proto UDP (17), length 51)
192.168.25.26.45683  192.168.25.3.53: [bad udp cksum 0xb39e -
0xf210!] 52886+ A? ya.ru. (23)
18:01:24.140544 IP (tos 0x0, ttl 64, id 17293, offset 0, flags [none],
proto UDP (17), length 51)
192.168.25.26.12575  192.168.25.3.53: [bad udp cksum 0xb39e -
0x7365!] 52886+ A? ya.ru. (23)

2015-07-07 16:09 GMT+03:00 Slawa Olhovchenkov s...@zxy.spb.ru:
 On Tue, Jul 07, 2015 at 02:50:49PM +0300, Pavel Timofeev wrote:

 Hi!
 I have a test virtual machine which runs CURRENT under Hyper-V. It's
 amd64 r285198 now.
 It can't get any response from MS DNS server. Well, it could two or
 three weeks ago, but after upgrade it's not able to do it anymore.
 Google DNS answers without problems meanwhile (sic!).

 What I do:
 # host google.ru 192.168.25.3
 I see that MS DNS (192.168.25.3) server receives these packets, but
 ignores them.
 And no matter how my system asks MS DNS. Every daemon can't get response too.

 I know that nothing was changed in MS DNS server. No doubt.
 Then I tried different available CURRENT snapshot ISOs.

 FreeBSD-11.0-CURRENT-amd64-20150630-r284969-disc1.iso - MS DNS does not 
 answer.

 FreeBSD-11.0-CURRENT-amd64-20150625-r284814-disc1.iso - MS DNS does not 
 answer.

 FreeBSD-11.0-CURRENT-amd64-20150618-r284544-disc1.iso - MS DNS answers!

 So something was committed to CURRENT between 20150618 and 20150625.
 This something ruins communication with MS DNS.

 Then I tried latest
 FreeBSD-11.0-CURRENT-amd64-20150630-r284969-disc1.iso on bare metal -
 MS DNS answered!

 Looks like that something is related to Hyper-V code.

 Maybe it changes packets somehow? I can gather and provide more info
 (tcpdump?) if you ask, it's not a problem!

 Author: whu
 Date: Wed Jun 24 06:01:29 2015
 New Revision: 284746
 URL: https://svnweb.freebsd.org/changeset/base/284746

 Log:
   TSO and checksum offloading support for Netvsc driver on Hyper-V.

 =

 Try tcpdump/wireshark on FreeBSD and MS DNS host.
 Check validating IP/UDP checksums.
 Try off checksum offloading on network interface
 (ifconfig ifname -txcsum -rxcsum)
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


vt(4) performance on Hyper-V

2014-08-18 Thread Pavel Timofeev
Hi!
Just in case, just so you know.
I've tried FreeBSD 10-STABLE r270125 with vt(4) on Hyper-V (Win Server
2012 DataCenter).
Its performance is extremely bad (while syscons is ok).
Here is video http://www.youtube.com/watch?v=62CIKCJ7bog

P.S. vt(4) is a bit faster on VirtualBox though when I tried it last time.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: vt(4) performance on Hyper-V

2014-08-18 Thread Pavel Timofeev
Wow, thank you! It helped.
Sorry for the noise, it's even written in man vt.

2014-08-18 12:25 GMT+04:00 Luigi Rizzo ri...@iet.unipi.it:



 On Mon, Aug 18, 2014 at 9:08 AM, Pavel Timofeev tim...@gmail.com wrote:

 Hi!
 Just in case, just so you know.
 I've tried FreeBSD 10-STABLE r270125 with vt(4) on Hyper-V (Win Server
 2012 DataCenter).
 Its performance is extremely bad (while syscons is ok).
 Here is video http://www.youtube.com/watch?v=62CIKCJ7bog


 i think you need to put the following in /boot/loader.conf

 hw.vga.textmode=1

 otherwise it is very slow even on bare metal.

 cheers
 luigi



 P.S. vt(4) is a bit faster on VirtualBox though when I tried it last time.
 ___
 freebsd-sta...@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-stable
 To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org




 --
 -+---
  Prof. Luigi RIZZO, ri...@iet.unipi.it  . Dip. di Ing. dell'Informazione
  http://www.iet.unipi.it/~luigi/. Universita` di Pisa
  TEL  +39-050-2211611   . via Diotisalvi 2
  Mobile   +39-338-6809875   . 56122 PISA (Italy)
 -+---
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: Running `sysctl -a` causes crash (Hyper-V)

2013-11-12 Thread Pavel Timofeev
I hope all these changes you have done will be included in FreeBSD 10-RELEASE.
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: Hyper-V

2013-11-12 Thread Pavel Timofeev
I figured out another operations that cause caclru message appearance,
such as 'top -n', car /proc/0/status, etc.
But I tried these on 9.2-RELEASE LIVE CD and I saw calcru messages.

So it seems like it's not related to brand new hyper-v integrated
components. It's related to whole FreeBSD system. Which part? I don't
know.

2013/11/12 Pavel Timofeev tim...@gmail.com:
 So, we catched the bug that caused crash. Thanks to all. Cool!

 As regards the messages about calcru: I tried to install CentOS 6.4
 on new virt machine on same host system. Linux doesn't have any
 strange messages about time.
 I tried to migrate FreeBSD and CentOS virt machines to another host
 which has another hardware. Same story.

 For example, try to boot beta-3 in LIVE CD mode. You won't see calcru
 messages. But if you try to run 'dclient hn0' you'll see calcru
 message storm for some time.
 On already installed system I can cause messages by doing
 service zabbix_server start (even if it's not allowed in rc.conf (!))

 But I can start mysql-server, zabbix_agentd, apache22 and it doesn't
 cause appearance of calcru messages.


 Any ideas? How and where can I dig?

 2013/11/6 Pavel Timofeev tim...@gmail.com:
 Small fix:
 kern.timecounter.hardware=TSC covered messages, but now it looks stange
 % time sleep 30
20.24 real 0.00 user 0.00 sys

 2013/11/6 Pavel Timofeev tim...@gmail.com:
 I already tried disabling Hyper-v time syncing, setting
 kern.timecounter.hardware to different type from
 kern.timecounter.choice, setting kern.hz to different values. But
 nothing helped.
 I'll try to install CentOS 6.4 on this machine and see what happens.

 2013/11/6 Pavel Timofeev tim...@gmail.com:
 Does anybody know how to disable Hyper-v time syncing it guest FreeBSD
 OS? I'd like to try this one.

 2013/11/6 Pavel Timofeev tim...@gmail.com:
 Thank you very much!
 I started to dig, and found that I can reboot my server running under
 Hyper-V by doing 'sysctl -a'.
 Can anybody confirm same behavoiur?

 FreeBSD 10.0-BETA3, Win 2012 Datacenter

 2013/11/5 Julian Elischer jul...@freebsd.org:
 On 11/4/13, 11:13 PM, Pavel Timofeev wrote:

 Hi! I upgraded to 10.0-BETA3 but still see messages like

 Tue Nov  5 11:09:05 MSK 2013
 calcru: runtime went backwards from 1728 usec to 889 usec for pid 981
 (zabbix_server)
 calcru: runtime went backwards from 1229 usec to 703 usec for pid 976
 (zabbix_server)
 calcru: runtime went backwards from 1019 usec to 595 usec for pid 978
 (zabbix_server)
 calcru: runtime went backwards from 3041 usec to 1611 usec for pid 969
 (zabbix_server)
 calcru: runtime went backwards from 1078 usec to 595 usec for pid 975
 (zabbix_server)
 calcru: runtime went backwards from 1114 usec to 617 usec for pid 980
 (zabbix_server)
 calcru: runtime went backwards from 1086 usec to 602 usec for pid 977
 (zabbix_server)
 calcru: runtime went backwards from 1195 usec to 782 usec for pid 970
 (zabbix_server)
 calcru: runtime went backwards from 3206 usec to 2157 usec for pid 979
 (zabbix_server)
 calcru: runtime went backwards from 2179 usec to 1542 usec for pid 972
 (zabbix_server)
 calcru: runtime went backwards from 1833 usec to 927 usec for pid 1026
 (cron)
 calcru: runtime went backwards from 1091 usec to 551 usec for pid 1022
 (sendmail)
 calcru: runtime went backwards from 823 usec to 416 usec for pid 1022
 (sendmail)
 calcru: runtime went backwards from 4691 usec to 2735 usec for pid
 1011 (sendmail)
 calcru: runtime went backwards from 1162 usec to 587 usec for pid 1011
 (sendmail)
 calcru: runtime went backwards from 947 usec to 478 usec for pid 1010
 (httpd)
 calcru: runtime went backwards from 895 usec to 452 usec for pid 1009
 (httpd)
 calcru: runtime went backwards from 954 usec to 482 usec for pid 1008
 (httpd)
 calcru: runtime went backwards from 909 usec to 459 usec for pid 1007
 (httpd)
 calcru: runtime went backwards from 1214 usec to 653 usec for pid 1006
 (httpd)
 calcru: runtime went backwards from 201793 usec to 102672 usec for pid
 996 (httpd)
 calcru: runtime went backwards from 1105 usec to 1064 usec for pid 991
 (sshd)
 calcru: runtime went backwards from 45196 usec to 23221 usec for pid
 974 (zabbix_server)
 calcru: runtime went backwards from 2192 usec to 1436 usec for pid 973
 (zabbix_server)
 calcru: runtime went backwards from 605 usec to 306 usec for pid 968
 (zabbix_server)
 calcru: runtime went backwards from 626 usec to 316 usec for pid 967
 (zabbix_server)
 calcru: runtime went backwards from 773 usec to 391 usec for pid 966
 (zabbix_server)
 calcru: runtime went backwards from 663 usec to 335 usec for pid 965
 (zabbix_server)
 calcru: runtime went backwards from 799 usec to 404 usec for pid 964
 (zabbix_server)
 calcru: runtime went backwards from 37723 usec to 19422 usec for pid
 963 (zabbix_server)
 calcru: runtime went backwards from 49425 usec to 26265 usec for pid
 962 (zabbix_server)
 calcru: runtime went backwards from 38883 usec to 20993 usec for pid
 961 (zabbix_server

Running `sysctl -a` causes crash (Hyper-V)

2013-11-11 Thread Pavel Timofeev
Hi! Can anybody confirm the subject?
Every time when I run `sysctl -a` get this

http://pastebin.com/Chf9qQTD

Additional
vmcore: http://yadi.sk/d/EAvZe3vZCVvp6
core.txt: http://yadi.sk/d/IBLJ8NVTCVvpn


It's on Windows Server 2012 Datacenter
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: Running `sysctl -a` causes crash (Hyper-V)

2013-11-11 Thread Pavel Timofeev
I'm sorry, could you explain how to do it? Which one is the 'core
file', vmcore or core.txt?

2013/11/11 Sergey Kandaurov pluk...@gmail.com:
 On 11 November 2013 16:38, Pavel Timofeev tim...@gmail.com wrote:
 Hi! Can anybody confirm the subject?
 Every time when I run `sysctl -a` get this

 http://pastebin.com/Chf9qQTD

 Can you show 'p vm_guest' from core file?

 --
 wbr,
 pluknet
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: Running `sysctl -a` causes crash (Hyper-V)

2013-11-11 Thread Pavel Timofeev
root@bsdtest:~ # kgdb /boot/kernel/kernel ./vmcore.1
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type show copying to see the conditions.
There is absolutely no warranty for GDB.  Type show warranty for details.
This GDB was configured as amd64-marcel-freebsd...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address   = 0x0
fault code  = supervisor read data, page not present
instruction pointer = 0x20:0x8095d79b
stack pointer   = 0x28:0xfe003d31c8f0
frame pointer   = 0x28:0xfe003d31c900
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 777 (sysctl)
trap number = 12
panic: page fault
cpuid = 1
KDB: stack backtrace:
#0 0x808e7a90 at kdb_backtrace+0x60
#1 0x808af575 at panic+0x155
#2 0x80c8e272 at trap_fatal+0x3a2
#3 0x80c8e549 at trap_pfault+0x2c9
#4 0x80c8dcd6 at trap+0x5e6
#5 0x80c74f72 at calltrap+0x8
#6 0x808ea529 at sysctl_kern_vm_guest+0x29
#7 0x808baacf at sysctl_root+0x24f
#8 0x808bb088 at userland_sysctl+0x1d8
#9 0x808bae74 at sys___sysctl+0x74
#10 0x80c8eb67 at amd64_syscall+0x357
#11 0x80c7525b at Xfast_syscall+0xfb
Uptime: 30s
Dumping 89 out of 1001 MB:..18%..36%..54%..72%..89%

Reading symbols from /boot/kernel/fuse.ko.symbols...done.
Loaded symbols for /boot/kernel/fuse.ko.symbols
Reading symbols from /boot/kernel/fdescfs.ko.symbols...done.
Loaded symbols for /boot/kernel/fdescfs.ko.symbols
#0  doadump (textdump=value optimized out) at pcpu.h:219
219 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) p vm_guest
$1 = 3
Current language:  auto; currently minimal
(kgdb)




2013/11/11 Sergey Kandaurov pluk...@gmail.com:
 [Please, don't top-post.]
 On 11 November 2013 17:06, Pavel Timofeev tim...@gmail.com wrote:
 I'm sorry, could you explain how to do it? Which one is the 'core
 file', vmcore or core.txt?


 It's vmcore.
 kgdb /path/to/kernel /path/to/vmcore

 2013/11/11 Sergey Kandaurov pluk...@gmail.com:
 On 11 November 2013 16:38, Pavel Timofeev tim...@gmail.com wrote:
 Hi! Can anybody confirm the subject?
 Every time when I run `sysctl -a` get this

 http://pastebin.com/Chf9qQTD

 Can you show 'p vm_guest' from core file?

 --
 wbr,
 pluknet
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: Running `sysctl -a` causes crash (Hyper-V)

2013-11-11 Thread Pavel Timofeev
We'll see, I started kernel rebuilding.

2013/11/11 Sergey Kandaurov pluk...@gmail.com:
 On 11 November 2013 18:13, Pavel Timofeev tim...@gmail.com wrote:
 [...]
 (kgdb) p vm_guest
 $1 = 3
 Current language:  auto; currently minimal
 (kgdb)


 What if you try this change?

 Index: sys/kern/subr_param.c
 ===
 --- sys/kern/subr_param.c(revision 257695)
 +++ sys/kern/subr_param.c(working copy)
 @@ -153,6 +153,7 @@
  none,
  generic,
  xen,
 +hv,
  NULL
  };
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: Hyper-V

2013-11-05 Thread Pavel Timofeev
Thank you very much!
I started to dig, and found that I can reboot my server running under
Hyper-V by doing 'sysctl -a'.
Can anybody confirm same behavoiur?

FreeBSD 10.0-BETA3, Win 2012 Datacenter

2013/11/5 Julian Elischer jul...@freebsd.org:
 On 11/4/13, 11:13 PM, Pavel Timofeev wrote:

 Hi! I upgraded to 10.0-BETA3 but still see messages like

 Tue Nov  5 11:09:05 MSK 2013
 calcru: runtime went backwards from 1728 usec to 889 usec for pid 981
 (zabbix_server)
 calcru: runtime went backwards from 1229 usec to 703 usec for pid 976
 (zabbix_server)
 calcru: runtime went backwards from 1019 usec to 595 usec for pid 978
 (zabbix_server)
 calcru: runtime went backwards from 3041 usec to 1611 usec for pid 969
 (zabbix_server)
 calcru: runtime went backwards from 1078 usec to 595 usec for pid 975
 (zabbix_server)
 calcru: runtime went backwards from 1114 usec to 617 usec for pid 980
 (zabbix_server)
 calcru: runtime went backwards from 1086 usec to 602 usec for pid 977
 (zabbix_server)
 calcru: runtime went backwards from 1195 usec to 782 usec for pid 970
 (zabbix_server)
 calcru: runtime went backwards from 3206 usec to 2157 usec for pid 979
 (zabbix_server)
 calcru: runtime went backwards from 2179 usec to 1542 usec for pid 972
 (zabbix_server)
 calcru: runtime went backwards from 1833 usec to 927 usec for pid 1026
 (cron)
 calcru: runtime went backwards from 1091 usec to 551 usec for pid 1022
 (sendmail)
 calcru: runtime went backwards from 823 usec to 416 usec for pid 1022
 (sendmail)
 calcru: runtime went backwards from 4691 usec to 2735 usec for pid
 1011 (sendmail)
 calcru: runtime went backwards from 1162 usec to 587 usec for pid 1011
 (sendmail)
 calcru: runtime went backwards from 947 usec to 478 usec for pid 1010
 (httpd)
 calcru: runtime went backwards from 895 usec to 452 usec for pid 1009
 (httpd)
 calcru: runtime went backwards from 954 usec to 482 usec for pid 1008
 (httpd)
 calcru: runtime went backwards from 909 usec to 459 usec for pid 1007
 (httpd)
 calcru: runtime went backwards from 1214 usec to 653 usec for pid 1006
 (httpd)
 calcru: runtime went backwards from 201793 usec to 102672 usec for pid
 996 (httpd)
 calcru: runtime went backwards from 1105 usec to 1064 usec for pid 991
 (sshd)
 calcru: runtime went backwards from 45196 usec to 23221 usec for pid
 974 (zabbix_server)
 calcru: runtime went backwards from 2192 usec to 1436 usec for pid 973
 (zabbix_server)
 calcru: runtime went backwards from 605 usec to 306 usec for pid 968
 (zabbix_server)
 calcru: runtime went backwards from 626 usec to 316 usec for pid 967
 (zabbix_server)
 calcru: runtime went backwards from 773 usec to 391 usec for pid 966
 (zabbix_server)
 calcru: runtime went backwards from 663 usec to 335 usec for pid 965
 (zabbix_server)
 calcru: runtime went backwards from 799 usec to 404 usec for pid 964
 (zabbix_server)
 calcru: runtime went backwards from 37723 usec to 19422 usec for pid
 963 (zabbix_server)
 calcru: runtime went backwards from 49425 usec to 26265 usec for pid
 962 (zabbix_server)
 calcru: runtime went backwards from 38883 usec to 20993 usec for pid
 961 (zabbix_server)
 calcru: runtime went backwards from 53478 usec to 27411 usec for pid
 960 (zabbix_server)
 calcru: runtime went backwards from 40845 usec to 21750 usec for pid
 959 (zabbix_server)
 calcru: runtime went backwards from 47835 usec to 24455 usec for pid
 958 (zabbix_server)
 calcru: runtime went backwards from 2257 usec to 1612 usec for pid 957
 (zabbix_server)
 calcru: runtime went backwards from 2361 usec to 1193 usec for pid 932
 (zabbix_agentd)
 calcru: runtime went backwards from 32053 usec to 16204 usec for pid
 928 (zabbix_server)
 calcru: runtime went backwards from 32517 usec to 19280 usec for pid 697
 (sh)
 calcru: runtime went backwards from 1128021 usec to 868022 usec for pid
 697 (sh)
 calcru: runtime went backwards from 9609 usec to 5601 usec for pid 601
 (syslogd)
 calcru: runtime went backwards from 688 usec to 348 usec for pid 106
 (adjkerntz)
 calcru: runtime went backwards from 12 usec to 6 usec for pid 3
 (sctp_iterator)
 calcru: runtime went backwards from 5467 usec to 2853 usec for pid 2
 (fdc0)
 calcru: runtime went backwards from 11306 usec to 8408 usec for pid 14
 (rand_harvestq)
 calcru: runtime went backwards from 136242 usec to 76230 usec for pid 13
 (geom)
 calcru: runtime went backwards from 156117 usec to 118882 usec for pid 12
 (intr)
 calcru: runtime went backwards from 7820 usec to 4847 usec for pid 1
 (init)
 calcru: runtime went backwards from 2340313 usec to 1226503 usec for
 pid 1 (init)
 calcru: runtime went backwards from 4114 usec to 2164 usec for pid 0
 (kernel)


 Is it bad? Or I have to ignore them?


 I'm not talking authoratatively but in some virtualization systems, if the
 TSCs on the various processors are not sufficiently synchronized, then when
 the hypervisor switches you from one CPU to another, you can get odd
 (sometimes negative) time readings which give messages like

Hyper-V

2013-11-04 Thread Pavel Timofeev
Hi! I upgraded to 10.0-BETA3 but still see messages like

Tue Nov  5 11:09:05 MSK 2013
calcru: runtime went backwards from 1728 usec to 889 usec for pid 981
(zabbix_server)
calcru: runtime went backwards from 1229 usec to 703 usec for pid 976
(zabbix_server)
calcru: runtime went backwards from 1019 usec to 595 usec for pid 978
(zabbix_server)
calcru: runtime went backwards from 3041 usec to 1611 usec for pid 969
(zabbix_server)
calcru: runtime went backwards from 1078 usec to 595 usec for pid 975
(zabbix_server)
calcru: runtime went backwards from 1114 usec to 617 usec for pid 980
(zabbix_server)
calcru: runtime went backwards from 1086 usec to 602 usec for pid 977
(zabbix_server)
calcru: runtime went backwards from 1195 usec to 782 usec for pid 970
(zabbix_server)
calcru: runtime went backwards from 3206 usec to 2157 usec for pid 979
(zabbix_server)
calcru: runtime went backwards from 2179 usec to 1542 usec for pid 972
(zabbix_server)
calcru: runtime went backwards from 1833 usec to 927 usec for pid 1026 (cron)
calcru: runtime went backwards from 1091 usec to 551 usec for pid 1022
(sendmail)
calcru: runtime went backwards from 823 usec to 416 usec for pid 1022 (sendmail)
calcru: runtime went backwards from 4691 usec to 2735 usec for pid
1011 (sendmail)
calcru: runtime went backwards from 1162 usec to 587 usec for pid 1011
(sendmail)
calcru: runtime went backwards from 947 usec to 478 usec for pid 1010 (httpd)
calcru: runtime went backwards from 895 usec to 452 usec for pid 1009 (httpd)
calcru: runtime went backwards from 954 usec to 482 usec for pid 1008 (httpd)
calcru: runtime went backwards from 909 usec to 459 usec for pid 1007 (httpd)
calcru: runtime went backwards from 1214 usec to 653 usec for pid 1006 (httpd)
calcru: runtime went backwards from 201793 usec to 102672 usec for pid
996 (httpd)
calcru: runtime went backwards from 1105 usec to 1064 usec for pid 991 (sshd)
calcru: runtime went backwards from 45196 usec to 23221 usec for pid
974 (zabbix_server)
calcru: runtime went backwards from 2192 usec to 1436 usec for pid 973
(zabbix_server)
calcru: runtime went backwards from 605 usec to 306 usec for pid 968
(zabbix_server)
calcru: runtime went backwards from 626 usec to 316 usec for pid 967
(zabbix_server)
calcru: runtime went backwards from 773 usec to 391 usec for pid 966
(zabbix_server)
calcru: runtime went backwards from 663 usec to 335 usec for pid 965
(zabbix_server)
calcru: runtime went backwards from 799 usec to 404 usec for pid 964
(zabbix_server)
calcru: runtime went backwards from 37723 usec to 19422 usec for pid
963 (zabbix_server)
calcru: runtime went backwards from 49425 usec to 26265 usec for pid
962 (zabbix_server)
calcru: runtime went backwards from 38883 usec to 20993 usec for pid
961 (zabbix_server)
calcru: runtime went backwards from 53478 usec to 27411 usec for pid
960 (zabbix_server)
calcru: runtime went backwards from 40845 usec to 21750 usec for pid
959 (zabbix_server)
calcru: runtime went backwards from 47835 usec to 24455 usec for pid
958 (zabbix_server)
calcru: runtime went backwards from 2257 usec to 1612 usec for pid 957
(zabbix_server)
calcru: runtime went backwards from 2361 usec to 1193 usec for pid 932
(zabbix_agentd)
calcru: runtime went backwards from 32053 usec to 16204 usec for pid
928 (zabbix_server)
calcru: runtime went backwards from 32517 usec to 19280 usec for pid 697 (sh)
calcru: runtime went backwards from 1128021 usec to 868022 usec for pid 697 (sh)
calcru: runtime went backwards from 9609 usec to 5601 usec for pid 601 (syslogd)
calcru: runtime went backwards from 688 usec to 348 usec for pid 106 (adjkerntz)
calcru: runtime went backwards from 12 usec to 6 usec for pid 3 (sctp_iterator)
calcru: runtime went backwards from 5467 usec to 2853 usec for pid 2 (fdc0)
calcru: runtime went backwards from 11306 usec to 8408 usec for pid 14
(rand_harvestq)
calcru: runtime went backwards from 136242 usec to 76230 usec for pid 13 (geom)
calcru: runtime went backwards from 156117 usec to 118882 usec for pid 12 (intr)
calcru: runtime went backwards from 7820 usec to 4847 usec for pid 1 (init)
calcru: runtime went backwards from 2340313 usec to 1226503 usec for
pid 1 (init)
calcru: runtime went backwards from 4114 usec to 2164 usec for pid 0 (kernel)


Is it bad? Or I have to ignore them?
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: Hyper-V driver development status

2013-08-29 Thread Pavel Timofeev
So, is it going to be included in FreeBSD 10 now? As I can see code
freeze for CURRENT is planned on 7 September 2013 (10 days from now).
http://www.freebsd.org/releases/10.0R/schedule.html
It's not much time.

2013/8/17 Abhishek Gupta (LIS) abgu...@microsoft.com:
 Hi Pavel,

 Thanks for inquiring about the BSD drivers for Hyper-V. We (Microsoft) are 
 actively working on these drivers and the intent is to get them integrated in 
 to the kernel within the next month or so. There are some issues we are 
 trying to address:

 a) Witness panic during boot time due to a specific memory allocation.
 b) Bypass usage of disk labels to ensure that devices do not get reordered 
 under FreeBSD.
 c) Locks up on 32 bit builds.
 d) Lack of KVP support for BSD drivers

 We are working hard every day to finish these items.

 Even though an official announcement has not gone out, I would highly 
 encourage you to start testing the drivers. As they are in project branch now 
 they should be easy to compile. The only requirement is to set up disk labels 
 before rebooting a kernel that includes the Hyper-V drivers otherwise the 
 system will not boot up.

 Please let me know if you have more questions or have feedback.

 Thanks again for your interest.
 Abhishek
 
 From: Peter Grehan gre...@freebsd.org
 Sent: Friday, August 16, 2013 11:30 AM
 To: Abhishek Gupta (LIS)
 Subject: Fwd: Hyper-V driver development status

 Want to reply to this ?


  Original Message 
 Subject: Hyper-V driver development status
 Date: Fri, 16 Aug 2013 17:00:57 +0400
 From: Pavel Timofeev tim...@gmail.com
 To: freebsd-virtualization@freebsd.org

 Hi all!
 I'd like to know what's happening with hyper-v driver development in
 FreeBSD.
 I know about svn branch but there is no active progress in there.
 What's the roadmap? Or we are waiting for something?

 What is the current status of that project?
 I could do some tests. Or it's too early?
 I'm looking forward to use it =)
 ___
 freebsd-virtualization@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
 To unsubscribe, send any mail to
 freebsd-virtualization-unsubscr...@freebsd.org



___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org