Re: freebsd downgrade

[Slawa Olhovchenkov]

On Tue, Jul 28, 2015 at 11:02:49PM +, Glen Barber wrote:

> On Wed, Jul 29, 2015 at 01:09:05AM +0300, Slawa Olhovchenkov wrote:
> > On Tue, Jul 28, 2015 at 09:45:28PM +, Glen Barber wrote:
> > 
> > > On Wed, Jul 29, 2015 at 12:42:12AM +0300, Slawa Olhovchenkov wrote:
> > > > On Sat, Jul 25, 2015 at 02:27:50AM +, Glen Barber wrote:
> > > > 
> > > > > On Fri, Jul 24, 2015 at 06:36:32PM +0300, Slawa Olhovchenkov wrote:
> > > > > > On Fri, Jul 24, 2015 at 08:25:55AM -0700, Navdeep Parhar wrote:
> > > > > > 
> > > > > > > On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov 
> > > > > > > wrote:
> > > > > > > > > > 
> > > > > > > > > > For me -- degradate network performance, about 30%. I am do
> > > > > > > > > > verification now, by downgrading kernel.
> > > > > > > > > 
> > > > > > > > > What kind of workload and what NIC are you using?
> > > > > > > > 
> > > > > > > > Chelsio T580-LP-CR, http serving.
> > > > > > > > r276179 can utilise 40Gbit, r281264 -- only 28Gbit.
> > > > > > > 
> > > > > > > Can you please try -RC1 when it comes out and report back if you 
> > > > > > > still
> > > > > > > see degraded network performance? 
> > > > > > 
> > > > > > Now I am try revisons r276179 and compare with r281264 (for 
> > > > > > absolutly
> > > > > > clean compare -- same hardware, same configuration, same workload)
> > > > > > before this I am comare slighty different hardware (card instaled in
> > > > > > different slots) and posibly different workload.
> > > > > > After this I am try r278800 and RC1 -- one times per day.
> > > > > > 
> > > > > 
> > > > > Please try with the 10.2-RC1 that was just released, and report back.
> > > > 
> > > > Root of cause is BIOS setting: need ACPI NUMA to OFF.
> > > > After this change performance gain about 30%.
> > > > Sorry for false alaram.
> > > > 
> > > 
> > > Thank you for providing this information.
> > 
> > Supermicro X10DRi + Dual E5-2640v3, if you interesting
> > 
> 
> Yes, thank you.
> 
> We have a few X10SLH-F/X10SLM+-F systems in the FreeBSD.org cluster, but
> I am unsure if this option is available on all X10-series systems.  I'll
> reboot one of them later today, and dig around in the BIOS.

For may workload (http servering) best result is

Hyper-Threading: OFF
Socket Interleave Below 4GB: ON
ACPI NUMA: OFF

For computing-intensive workload may be differ.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Glen Barber]

On Wed, Jul 29, 2015 at 01:09:05AM +0300, Slawa Olhovchenkov wrote:
> On Tue, Jul 28, 2015 at 09:45:28PM +, Glen Barber wrote:
> 
> > On Wed, Jul 29, 2015 at 12:42:12AM +0300, Slawa Olhovchenkov wrote:
> > > On Sat, Jul 25, 2015 at 02:27:50AM +, Glen Barber wrote:
> > > 
> > > > On Fri, Jul 24, 2015 at 06:36:32PM +0300, Slawa Olhovchenkov wrote:
> > > > > On Fri, Jul 24, 2015 at 08:25:55AM -0700, Navdeep Parhar wrote:
> > > > > 
> > > > > > On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov wrote:
> > > > > > > > > 
> > > > > > > > > For me -- degradate network performance, about 30%. I am do
> > > > > > > > > verification now, by downgrading kernel.
> > > > > > > > 
> > > > > > > > What kind of workload and what NIC are you using?
> > > > > > > 
> > > > > > > Chelsio T580-LP-CR, http serving.
> > > > > > > r276179 can utilise 40Gbit, r281264 -- only 28Gbit.
> > > > > > 
> > > > > > Can you please try -RC1 when it comes out and report back if you 
> > > > > > still
> > > > > > see degraded network performance? 
> > > > > 
> > > > > Now I am try revisons r276179 and compare with r281264 (for absolutly
> > > > > clean compare -- same hardware, same configuration, same workload)
> > > > > before this I am comare slighty different hardware (card instaled in
> > > > > different slots) and posibly different workload.
> > > > > After this I am try r278800 and RC1 -- one times per day.
> > > > > 
> > > > 
> > > > Please try with the 10.2-RC1 that was just released, and report back.
> > > 
> > > Root of cause is BIOS setting: need ACPI NUMA to OFF.
> > > After this change performance gain about 30%.
> > > Sorry for false alaram.
> > > 
> > 
> > Thank you for providing this information.
> 
> Supermicro X10DRi + Dual E5-2640v3, if you interesting
> 

Yes, thank you.

We have a few X10SLH-F/X10SLM+-F systems in the FreeBSD.org cluster, but
I am unsure if this option is available on all X10-series systems.  I'll
reboot one of them later today, and dig around in the BIOS.

Glen



pgpaHWmqMgTUB.pgp
Description: PGP signature

Re: freebsd downgrade

[Slawa Olhovchenkov]

On Tue, Jul 28, 2015 at 09:45:28PM +, Glen Barber wrote:

> On Wed, Jul 29, 2015 at 12:42:12AM +0300, Slawa Olhovchenkov wrote:
> > On Sat, Jul 25, 2015 at 02:27:50AM +, Glen Barber wrote:
> > 
> > > On Fri, Jul 24, 2015 at 06:36:32PM +0300, Slawa Olhovchenkov wrote:
> > > > On Fri, Jul 24, 2015 at 08:25:55AM -0700, Navdeep Parhar wrote:
> > > > 
> > > > > On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov wrote:
> > > > > > > > 
> > > > > > > > For me -- degradate network performance, about 30%. I am do
> > > > > > > > verification now, by downgrading kernel.
> > > > > > > 
> > > > > > > What kind of workload and what NIC are you using?
> > > > > > 
> > > > > > Chelsio T580-LP-CR, http serving.
> > > > > > r276179 can utilise 40Gbit, r281264 -- only 28Gbit.
> > > > > 
> > > > > Can you please try -RC1 when it comes out and report back if you still
> > > > > see degraded network performance? 
> > > > 
> > > > Now I am try revisons r276179 and compare with r281264 (for absolutly
> > > > clean compare -- same hardware, same configuration, same workload)
> > > > before this I am comare slighty different hardware (card instaled in
> > > > different slots) and posibly different workload.
> > > > After this I am try r278800 and RC1 -- one times per day.
> > > > 
> > > 
> > > Please try with the 10.2-RC1 that was just released, and report back.
> > 
> > Root of cause is BIOS setting: need ACPI NUMA to OFF.
> > After this change performance gain about 30%.
> > Sorry for false alaram.
> > 
> 
> Thank you for providing this information.

Supermicro X10DRi + Dual E5-2640v3, if you interesting
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Glen Barber]

On Wed, Jul 29, 2015 at 12:42:12AM +0300, Slawa Olhovchenkov wrote:
> On Sat, Jul 25, 2015 at 02:27:50AM +, Glen Barber wrote:
> 
> > On Fri, Jul 24, 2015 at 06:36:32PM +0300, Slawa Olhovchenkov wrote:
> > > On Fri, Jul 24, 2015 at 08:25:55AM -0700, Navdeep Parhar wrote:
> > > 
> > > > On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov wrote:
> > > > > > > 
> > > > > > > For me -- degradate network performance, about 30%. I am do
> > > > > > > verification now, by downgrading kernel.
> > > > > > 
> > > > > > What kind of workload and what NIC are you using?
> > > > > 
> > > > > Chelsio T580-LP-CR, http serving.
> > > > > r276179 can utilise 40Gbit, r281264 -- only 28Gbit.
> > > > 
> > > > Can you please try -RC1 when it comes out and report back if you still
> > > > see degraded network performance? 
> > > 
> > > Now I am try revisons r276179 and compare with r281264 (for absolutly
> > > clean compare -- same hardware, same configuration, same workload)
> > > before this I am comare slighty different hardware (card instaled in
> > > different slots) and posibly different workload.
> > > After this I am try r278800 and RC1 -- one times per day.
> > > 
> > 
> > Please try with the 10.2-RC1 that was just released, and report back.
> 
> Root of cause is BIOS setting: need ACPI NUMA to OFF.
> After this change performance gain about 30%.
> Sorry for false alaram.
> 

Thank you for providing this information.

Glen



pgphCSZuM4juN.pgp
Description: PGP signature

Re: freebsd downgrade

[Slawa Olhovchenkov]

On Sat, Jul 25, 2015 at 02:27:50AM +, Glen Barber wrote:

> On Fri, Jul 24, 2015 at 06:36:32PM +0300, Slawa Olhovchenkov wrote:
> > On Fri, Jul 24, 2015 at 08:25:55AM -0700, Navdeep Parhar wrote:
> > 
> > > On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov wrote:
> > > > > > 
> > > > > > For me -- degradate network performance, about 30%. I am do
> > > > > > verification now, by downgrading kernel.
> > > > > 
> > > > > What kind of workload and what NIC are you using?
> > > > 
> > > > Chelsio T580-LP-CR, http serving.
> > > > r276179 can utilise 40Gbit, r281264 -- only 28Gbit.
> > > 
> > > Can you please try -RC1 when it comes out and report back if you still
> > > see degraded network performance? 
> > 
> > Now I am try revisons r276179 and compare with r281264 (for absolutly
> > clean compare -- same hardware, same configuration, same workload)
> > before this I am comare slighty different hardware (card instaled in
> > different slots) and posibly different workload.
> > After this I am try r278800 and RC1 -- one times per day.
> > 
> 
> Please try with the 10.2-RC1 that was just released, and report back.

Root of cause is BIOS setting: need ACPI NUMA to OFF.
After this change performance gain about 30%.
Sorry for false alaram.


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Glen Barber]

On Fri, Jul 24, 2015 at 06:36:32PM +0300, Slawa Olhovchenkov wrote:
> On Fri, Jul 24, 2015 at 08:25:55AM -0700, Navdeep Parhar wrote:
> 
> > On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov wrote:
> > > > > 
> > > > > For me -- degradate network performance, about 30%. I am do
> > > > > verification now, by downgrading kernel.
> > > > 
> > > > What kind of workload and what NIC are you using?
> > > 
> > > Chelsio T580-LP-CR, http serving.
> > > r276179 can utilise 40Gbit, r281264 -- only 28Gbit.
> > 
> > Can you please try -RC1 when it comes out and report back if you still
> > see degraded network performance? 
> 
> Now I am try revisons r276179 and compare with r281264 (for absolutly
> clean compare -- same hardware, same configuration, same workload)
> before this I am comare slighty different hardware (card instaled in
> different slots) and posibly different workload.
> After this I am try r278800 and RC1 -- one times per day.
> 

Please try with the 10.2-RC1 that was just released, and report back.

Thanks.

Glen



pgpuOMFYU9SAj.pgp
Description: PGP signature

Re: freebsd downgrade

[Slawa Olhovchenkov]

On Fri, Jul 24, 2015 at 08:25:55AM -0700, Navdeep Parhar wrote:

> On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov wrote:
> > > > 
> > > > For me -- degradate network performance, about 30%. I am do
> > > > verification now, by downgrading kernel.
> > > 
> > > What kind of workload and what NIC are you using?
> > 
> > Chelsio T580-LP-CR, http serving.
> > r276179 can utilise 40Gbit, r281264 -- only 28Gbit.
> 
> Can you please try -RC1 when it comes out and report back if you still
> see degraded network performance? 

Now I am try revisons r276179 and compare with r281264 (for absolutly
clean compare -- same hardware, same configuration, same workload)
before this I am comare slighty different hardware (card instaled in
different slots) and posibly different workload.
After this I am try r278800 and RC1 -- one times per day.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Navdeep Parhar]

On Fri, Jul 24, 2015 at 10:50:10AM +0300, Slawa Olhovchenkov wrote:
> > > 
> > > For me -- degradate network performance, about 30%. I am do
> > > verification now, by downgrading kernel.
> > 
> > What kind of workload and what NIC are you using?
> 
> Chelsio T580-LP-CR, http serving.
> r276179 can utilise 40Gbit, r281264 -- only 28Gbit.

Can you please try -RC1 when it comes out and report back if you still
see degraded network performance? 

Regards,
Navdeep
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Walter Hop]

> I had a problems with FreeBSD 10.2-BETA2. I used freebsd-update ugrade
> and update FreeBSD 10.1-RELEASE.
> Is it possible to downgrade BETA2 to 10.1-RELEASE with freebsd-update
> upgrade -r 10.1-RELEASE, please?

In my experience, downgrading with freebsd-update works fine, although I only 
went back from 10.1 to 9.3, but arguably that’s a much bigger jump. We’ve done 
it on around 40 machines over ssh. You just have to skip the initial reboot, in 
order to stay in the new kernel. Here is a blog post I wrote about it: 
https://lifeforms.nl/20141224/downgrading-with-freebsd-update/ 


I can’t guarantee that going back from 10.2 to 10.1 will work too; there might 
be problems, but if you try it, let me know your experiences and I’ll update 
the post.

-- 
Walter Hop | PGP key: https://lifeforms.nl/pgp

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Stari Karp]

On Thu, 2015-07-23 at 16:51 -0700, Xin Li wrote:
> On 07/23/15 16:21, Stari Karp wrote:
> > Hi!
> > 
> > I had a problems with FreeBSD 10.2-BETA2. I used freebsd-update
> > ugrade and update FreeBSD 10.1-RELEASE. Is it possible to downgrade
> > BETA2 to 10.1-RELEASE with freebsd-update upgrade -r 10.1-RELEASE,
> > please?
> 
> It's not supported (because you may be running binaries that depends
> on new kernel).
> 
> What kind of problems did you have?  Please let re@ know so we can get
> them fixed.
> 
> Cheers,

I have a post in FreeBSD Stable maling list about my problem. A subject
is: FreeBSD 10.1_RELEASE to FreeBSD 10.2 BETA2

And I wrote:
" Hi!

I upgrade 10.1-RELEASE (amd64) with freebsd-update upgrade to
 FreeBSD 10.2-BETA2 #0 r285646: Fri Jul 17 01:09:28 UTC 2015
r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64

and now I have some problems:

1. VT module doesn't load. 
   In /boot/loader.conf I have:
   kern.vty=vt
but it doesn't work. If I manual loada module witk kldload radeonkms I
got just black screen and nothing more.

2. Fireware external disk doesn't work either. In /boot/loader.conf I
have sbp_load="YES" but it doesn't load. If I run kldload sbp it load
and works.
3. The same is with web camera.
   cuse4bsd_load="YES" which I have in /boot/loader.conf doesn't load.
It works if I load with kldload.
In /boot/loader.conf I have also loader_logo="beastie" which doesn't
show picture in menu (it is default one).

In /boot/device.hints I have also some hits which are for my sound card
and it doesn't read them and sound doesn't work correct too.

On FreeBSD 10.1-RELEASE (amd 64) I didn't have problems."

 

Thank you.


___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Slawa Olhovchenkov]

On Fri, Jul 24, 2015 at 12:43:11AM -0700, Xin Li wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> 
> 
> On 7/24/15 00:20, Slawa Olhovchenkov wrote:
> > On Thu, Jul 23, 2015 at 04:51:07PM -0700, Xin Li wrote:
> > 
> >> -BEGIN PGP SIGNED MESSAGE- Hash: SHA512
> >> 
> >> On 07/23/15 16:21, Stari Karp wrote:
> >>> Hi!
> >>> 
> >>> I had a problems with FreeBSD 10.2-BETA2. I used
> >>> freebsd-update ugrade and update FreeBSD 10.1-RELEASE. Is it
> >>> possible to downgrade BETA2 to 10.1-RELEASE with freebsd-update
> >>> upgrade -r 10.1-RELEASE, please?
> >> 
> >> It's not supported (because you may be running binaries that
> >> depends on new kernel).
> >> 
> >> What kind of problems did you have?  Please let re@ know so we
> >> can get them fixed.
> > 
> > For me -- degradate network performance, about 30%. I am do
> > verification now, by downgrading kernel.
> 
> What kind of workload and what NIC are you using?

Chelsio T580-LP-CR, http serving.
r276179 can utilise 40Gbit, r281264 -- only 28Gbit.

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Xin Li]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512



On 7/24/15 00:20, Slawa Olhovchenkov wrote:
> On Thu, Jul 23, 2015 at 04:51:07PM -0700, Xin Li wrote:
> 
>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA512
>> 
>> On 07/23/15 16:21, Stari Karp wrote:
>>> Hi!
>>> 
>>> I had a problems with FreeBSD 10.2-BETA2. I used
>>> freebsd-update ugrade and update FreeBSD 10.1-RELEASE. Is it
>>> possible to downgrade BETA2 to 10.1-RELEASE with freebsd-update
>>> upgrade -r 10.1-RELEASE, please?
>> 
>> It's not supported (because you may be running binaries that
>> depends on new kernel).
>> 
>> What kind of problems did you have?  Please let re@ know so we
>> can get them fixed.
> 
> For me -- degradate network performance, about 30%. I am do
> verification now, by downgrading kernel.

What kind of workload and what NIC are you using?

Cheers,
-BEGIN PGP SIGNATURE-

iQIcBAEBCgAGBQJVseyOAAoJEJW2GBstM+nsn5YP/jjX9IM2T7wQ8p5skZDvA5e+
e/dZ+IWKsqOIh612MyXfYbMdHbKakXdMP8K2rU6aNQmTinArSbC/xhUPXnA7029y
Xz4TLTyb2y/J1oNFTU7saSnv5CeB8n/oDGxCISDqJPZDK9rypBjO7r3kMP9OLc5R
c9Qtd38i2mdH3oWHbDeonBNaJ5YQ32zB5nGWEMpxFc2xQU2GK37dx4xjCJdx3L+1
WS+s2FJdWXCsclE12luHiGjD1hlIIS4yLZhSgLUy9i+rD4Zn6VOF7QvwCIAbjD8h
kpZDt9YcqJy7y/gZu2iFa12nIguY9Z6pmPHu8yy/lGztfmspUWnmDspYnBhDJyoi
GHrCji5O7St8w1MCUPYPsE/crvcel16FA3W1yCmYtmzX+HyEjcuqNY41vN3WqkCg
thYyJxS5IVSXHFR3wbfsNM3WlG6pREE0uzjqo4rbzxRNPGHzsmoD8ImOUa3pKjgP
XwgPSv8J7I4uskW6isDWuB5GBmsPk4PqZfUWvnCqYmG77HX5sl0ct+gxQQ+b6sAl
6I2FYflvrvexRhxDMc+gjoArNDOJhSVQY/fqswQXdsPYdIQnhmKLCX+G3ZOx0cho
uYN6Qf6bqcSAcwKZeRI5LAOEc1XpqEo8HhxkwKgmC7m3SWJs35m9+yom7PlPdyM7
vx/huCTRA26XXuNAxqJF
=Mhdf
-END PGP SIGNATURE-
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Slawa Olhovchenkov]

On Thu, Jul 23, 2015 at 04:51:07PM -0700, Xin Li wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> On 07/23/15 16:21, Stari Karp wrote:
> > Hi!
> > 
> > I had a problems with FreeBSD 10.2-BETA2. I used freebsd-update
> > ugrade and update FreeBSD 10.1-RELEASE. Is it possible to downgrade
> > BETA2 to 10.1-RELEASE with freebsd-update upgrade -r 10.1-RELEASE,
> > please?
> 
> It's not supported (because you may be running binaries that depends
> on new kernel).
> 
> What kind of problems did you have?  Please let re@ know so we can get
> them fixed.

For me -- degradate network performance, about 30%.
I am do verification now, by downgrading kernel.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: freebsd downgrade

[Xin Li]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 07/23/15 16:21, Stari Karp wrote:
> Hi!
> 
> I had a problems with FreeBSD 10.2-BETA2. I used freebsd-update
> ugrade and update FreeBSD 10.1-RELEASE. Is it possible to downgrade
> BETA2 to 10.1-RELEASE with freebsd-update upgrade -r 10.1-RELEASE,
> please?

It's not supported (because you may be running binaries that depends
on new kernel).

What kind of problems did you have?  Please let re@ know so we can get
them fixed.

Cheers,
- -- 
Xin LI https://www.delphij.net/
FreeBSD - The Power to Serve!   Live free or die
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.1.6 (FreeBSD)

iQIcBAEBCgAGBQJVsX3rAAoJEJW2GBstM+nshTsP/1gkFDhlrzElnOPfxtQEc80H
6oocZG4Upn9UbrFIq9GKU4qU8XIC8fHtYtLyXAbjj5vebEMkOAJuhzE8HrN3eh+N
hZKtKRiK7fNSFWCkqmd2BY0G05IBEXgQXzbgRzc65OrRcrp5MSL109bxYjN7Ih1L
wMtulWxfxyqsCZ4h6qwjZehJTkPMwiaGfhjbxAUbghzJKEyCKVGmGCE5Wbz4q++W
9bkQXElu1777WlJ3X+6tvgRkBxpYdaXSzL2UAH5cZrM7b5T09KfcgKsl+x9o1CGN
PGRQZqNcdyxEebjMqJaE2BOuvtocIrlpEOtC0VT7f5pLp64pV8PYPQGzdi+01LZD
w/wNvkJFqhIT42h/DDuTj7kOsrJhDA5Q2Vho9KW4YZFbDO9eglCA2KhDgrLQejl3
9z0byDWq+my638CRQIjHcf0QBokdXceK/AVjnIx9Os59OD0dQkl3ISSRhSDQPmqe
urI96uQ582vo6h+Tiic119aPtmy7aTn6PqtsTEYLnb5YTFx8oF4XaBuP7a3RmcBm
JX3oX5vIzOiDWtnG27yhsy5Bido91xPznnCLoC24MKuO7YFJb9Pi4x4fy8jIQkpp
gZTz7AkGR8oFroisPb9DGkkhT5QcwQviEnyRA1pR0dk7Sv6mzckRRSxxMn6s1SrY
0oVGHIe54nF7T1MRZc15
=BS7Y
-END PGP SIGNATURE-
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"