Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-05-04 Thread Pasi Kärkkäinen 
On Wed, Jan 24, 2018 at 09:30:33AM -0800, Nathan March wrote:
> 
> > -Original Message-
> > From: CentOS-virt [mailto:centos-virt-boun...@centos.org] On Behalf Of
> > Johnny Hughes
> > Sent: Wednesday, January 24, 2018 6:39 AM
> > To: centos-virt@centos.org
> > Subject: Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation)
> > packages making their way to centos-virt-xen-testing
> > 
> > On 01/24/2018 01:01 AM, Pasi Kärkkäinen wrote:
> > > On Tue, Jan 23, 2018 at 06:20:39PM -0600, Kevin Stange wrote:
> > >> On 01/23/2018 05:57 PM, Karl Johnson wrote:
> > >>>
> > >>>
> > >>> On Tue, Jan 23, 2018 at 4:50 PM, Nathan March  > >>> <mailto:nat...@gt.net>> wrote:
> > >>>
> > >>> Hi,
> > >>>
> > >>> > Hmm.. isn't this the ldisc bug that was discussed a few months ago
> > on this
> > >>> list,
> > >>> > and a patch was applied to virt-sig kernel aswell?
> > >>> >
> > >>> > Call trace looks similar..
> > >>>
> > >>> Good memory! I'd forgotten about that despite being the one who ran
> > >>> into it.
> > >>>
> > >>> Looks like that patch was just removed in 4.9.75-30 which I just
> > >>> upgraded
> > >>> this system to: http://cbs.centos.org/koji/buildinfo?buildID=21122
> > >>> <http://cbs.centos.org/koji/buildinfo?buildID=21122>
> > >>> Previously I was on 4.9.63-29 which does not have this problem, and
> > does
> > >>> have the ldisc patch. So I guess the question is for Johnny, why 
> > >>> was it
> > >>> removed?
> > >>>
> > >>> In the meantime, I'll revert the kernel and follow up if I see any
> > >>> further
> > >>> problems.
> > >>>
> > >>>
> > >>> IIRC the patch has been removed from the spec file because it has been
> > >>> merged upstream in 4.9.71.
> > >>
> > >> The IRC discussion I found in my log indicates that it was removed
> > >> because it didn't apply cleanly due to changes when updating to 4.9.75,
> > >> yet I don't think anyone independently validated that the changes made
> > >> are equivalent to the patch that was removed.  I was never able to
> > >> reproduce this issue, so I didn't investigate it myself.
> > >>
> > >
> > > Sounds like the patch is still needed :)
> > >
> > > Anyone up to re-porting it to 4.9.75+ ?
> > 
> > It looked, at first glance, like 4.9.71 fixed it .. I guess not in all cases
> 
> I'm happy to do testing here if anyone's able to help with a patch, does look 
> like reverting to 4.9.63-29 solved it for me in the interm.
> 

Hello Nathan,

I noticed Upstream Linux now has some patches related to tty_ldisc .. as you 
seem to have an easily reproducible way of triggering the crashes, could you 
please try if these patches fix the issue, when applied to CentOS 4.9 kernel?


"tty: fix data race between tty_init_dev and flush of buf":
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/tty/tty_ldisc.c?id=b027e2298bd588d6fa36ed2eda97447fb3eac078

"tty: Avoid possible error pointer dereference at tty_ldisc_restore().":
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/tty/tty_ldisc.c?id=598c2d41ff44889dd8eced4f117403e472158d85

"tty: Don't call panic() at tty_ldisc_init()":
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/tty/tty_ldisc.c?id=903f9db10f18f735e62ba447147b6c434b6af003

"tty: Use __GFP_NOFAIL for tty_ldisc_get()":
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/tty/tty_ldisc.c?id=bcdd0ca8cb8730573afebcaae4138f8f4c8eaa20


Thanks,

-- Pasi

___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-24 Thread Nathan March 

> -Original Message-
> From: CentOS-virt [mailto:centos-virt-boun...@centos.org] On Behalf Of
> Johnny Hughes
> Sent: Wednesday, January 24, 2018 6:39 AM
> To: centos-virt@centos.org
> Subject: Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation)
> packages making their way to centos-virt-xen-testing
> 
> On 01/24/2018 01:01 AM, Pasi Kärkkäinen wrote:
> > On Tue, Jan 23, 2018 at 06:20:39PM -0600, Kevin Stange wrote:
> >> On 01/23/2018 05:57 PM, Karl Johnson wrote:
> >>>
> >>>
> >>> On Tue, Jan 23, 2018 at 4:50 PM, Nathan March  >>> <mailto:nat...@gt.net>> wrote:
> >>>
> >>> Hi,
> >>>
> >>> > Hmm.. isn't this the ldisc bug that was discussed a few months ago
> on this
> >>> list,
> >>> > and a patch was applied to virt-sig kernel aswell?
> >>> >
> >>> > Call trace looks similar..
> >>>
> >>> Good memory! I'd forgotten about that despite being the one who ran
> >>> into it.
> >>>
> >>> Looks like that patch was just removed in 4.9.75-30 which I just
> >>> upgraded
> >>> this system to: http://cbs.centos.org/koji/buildinfo?buildID=21122
> >>> <http://cbs.centos.org/koji/buildinfo?buildID=21122>
> >>> Previously I was on 4.9.63-29 which does not have this problem, and
> does
> >>> have the ldisc patch. So I guess the question is for Johnny, why was 
> >>> it
> >>> removed?
> >>>
> >>> In the meantime, I'll revert the kernel and follow up if I see any
> >>> further
> >>> problems.
> >>>
> >>>
> >>> IIRC the patch has been removed from the spec file because it has been
> >>> merged upstream in 4.9.71.
> >>
> >> The IRC discussion I found in my log indicates that it was removed
> >> because it didn't apply cleanly due to changes when updating to 4.9.75,
> >> yet I don't think anyone independently validated that the changes made
> >> are equivalent to the patch that was removed.  I was never able to
> >> reproduce this issue, so I didn't investigate it myself.
> >>
> >
> > Sounds like the patch is still needed :)
> >
> > Anyone up to re-porting it to 4.9.75+ ?
> 
> It looked, at first glance, like 4.9.71 fixed it .. I guess not in all cases

I'm happy to do testing here if anyone's able to help with a patch, does look 
like reverting to 4.9.63-29 solved it for me in the interm.


___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-24 Thread Johnny Hughes 
On 01/24/2018 01:01 AM, Pasi Kärkkäinen wrote:
> On Tue, Jan 23, 2018 at 06:20:39PM -0600, Kevin Stange wrote:
>> On 01/23/2018 05:57 PM, Karl Johnson wrote:
>>>
>>>
>>> On Tue, Jan 23, 2018 at 4:50 PM, Nathan March >> > wrote:
>>>
>>> Hi,
>>>
>>> > Hmm.. isn't this the ldisc bug that was discussed a few months ago on 
>>> this
>>> list,
>>> > and a patch was applied to virt-sig kernel aswell?
>>> >
>>> > Call trace looks similar..
>>>
>>> Good memory! I'd forgotten about that despite being the one who ran
>>> into it.
>>>
>>> Looks like that patch was just removed in 4.9.75-30 which I just
>>> upgraded
>>> this system to: http://cbs.centos.org/koji/buildinfo?buildID=21122
>>> 
>>> Previously I was on 4.9.63-29 which does not have this problem, and does
>>> have the ldisc patch. So I guess the question is for Johnny, why was it
>>> removed?
>>>
>>> In the meantime, I'll revert the kernel and follow up if I see any
>>> further
>>> problems.
>>>
>>>
>>> IIRC the patch has been removed from the spec file because it has been
>>> merged upstream in 4.9.71.
>>
>> The IRC discussion I found in my log indicates that it was removed
>> because it didn't apply cleanly due to changes when updating to 4.9.75,
>> yet I don't think anyone independently validated that the changes made
>> are equivalent to the patch that was removed.  I was never able to
>> reproduce this issue, so I didn't investigate it myself.
>>
> 
> Sounds like the patch is still needed :)
> 
> Anyone up to re-porting it to 4.9.75+ ?

It looked, at first glance, like 4.9.71 fixed it .. I guess not in all cases





signature.asc
Description: OpenPGP digital signature
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-23 Thread Pasi Kärkkäinen 
On Tue, Jan 23, 2018 at 06:20:39PM -0600, Kevin Stange wrote:
> On 01/23/2018 05:57 PM, Karl Johnson wrote:
> > 
> > 
> > On Tue, Jan 23, 2018 at 4:50 PM, Nathan March  > > wrote:
> > 
> > Hi,
> > 
> > > Hmm.. isn't this the ldisc bug that was discussed a few months ago on 
> > this
> > list,
> > > and a patch was applied to virt-sig kernel aswell?
> > >
> > > Call trace looks similar..
> > 
> > Good memory! I'd forgotten about that despite being the one who ran
> > into it.
> > 
> > Looks like that patch was just removed in 4.9.75-30 which I just
> > upgraded
> > this system to: http://cbs.centos.org/koji/buildinfo?buildID=21122
> > 
> > Previously I was on 4.9.63-29 which does not have this problem, and does
> > have the ldisc patch. So I guess the question is for Johnny, why was it
> > removed?
> > 
> > In the meantime, I'll revert the kernel and follow up if I see any
> > further
> > problems.
> > 
> > 
> > IIRC the patch has been removed from the spec file because it has been
> > merged upstream in 4.9.71.
> 
> The IRC discussion I found in my log indicates that it was removed
> because it didn't apply cleanly due to changes when updating to 4.9.75,
> yet I don't think anyone independently validated that the changes made
> are equivalent to the patch that was removed.  I was never able to
> reproduce this issue, so I didn't investigate it myself.
>

Sounds like the patch is still needed :)

Anyone up to re-porting it to 4.9.75+ ?


-- Pasi
 
> -- 
> Kevin Stange
> Chief Technology Officer
> Steadfast | Managed Infrastructure, Datacenter and Cloud Services
> 800 S Wells, Suite 190 | Chicago, IL 60607
> 312.602.2689 X203 | Fax: 312.602.2688
> ke...@steadfast.net | www.steadfast.net
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-23 Thread Kevin Stange 
On 01/23/2018 05:57 PM, Karl Johnson wrote:
> 
> 
> On Tue, Jan 23, 2018 at 4:50 PM, Nathan March  > wrote:
> 
> Hi,
> 
> > Hmm.. isn't this the ldisc bug that was discussed a few months ago on 
> this
> list,
> > and a patch was applied to virt-sig kernel aswell?
> >
> > Call trace looks similar..
> 
> Good memory! I'd forgotten about that despite being the one who ran
> into it.
> 
> Looks like that patch was just removed in 4.9.75-30 which I just
> upgraded
> this system to: http://cbs.centos.org/koji/buildinfo?buildID=21122
> 
> Previously I was on 4.9.63-29 which does not have this problem, and does
> have the ldisc patch. So I guess the question is for Johnny, why was it
> removed?
> 
> In the meantime, I'll revert the kernel and follow up if I see any
> further
> problems.
> 
> 
> IIRC the patch has been removed from the spec file because it has been
> merged upstream in 4.9.71.

The IRC discussion I found in my log indicates that it was removed
because it didn't apply cleanly due to changes when updating to 4.9.75,
yet I don't think anyone independently validated that the changes made
are equivalent to the patch that was removed.  I was never able to
reproduce this issue, so I didn't investigate it myself.

-- 
Kevin Stange
Chief Technology Officer
Steadfast | Managed Infrastructure, Datacenter and Cloud Services
800 S Wells, Suite 190 | Chicago, IL 60607
312.602.2689 X203 | Fax: 312.602.2688
ke...@steadfast.net | www.steadfast.net
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-23 Thread Karl Johnson 
On Tue, Jan 23, 2018 at 4:50 PM, Nathan March  wrote:

> Hi,
>
> > Hmm.. isn't this the ldisc bug that was discussed a few months ago on
> this
> list,
> > and a patch was applied to virt-sig kernel aswell?
> >
> > Call trace looks similar..
>
> Good memory! I'd forgotten about that despite being the one who ran into
> it.
>
> Looks like that patch was just removed in 4.9.75-30 which I just upgraded
> this system to: http://cbs.centos.org/koji/buildinfo?buildID=21122
> Previously I was on 4.9.63-29 which does not have this problem, and does
> have the ldisc patch. So I guess the question is for Johnny, why was it
> removed?
>
> In the meantime, I'll revert the kernel and follow up if I see any further
> problems.
>
>
IIRC the patch has been removed from the spec file because it has been
merged upstream in 4.9.71.

Karl
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-23 Thread Nathan March 
Hi,

> Hmm.. isn't this the ldisc bug that was discussed a few months ago on this
list,
> and a patch was applied to virt-sig kernel aswell?
> 
> Call trace looks similar..

Good memory! I'd forgotten about that despite being the one who ran into it.

Looks like that patch was just removed in 4.9.75-30 which I just upgraded
this system to: http://cbs.centos.org/koji/buildinfo?buildID=21122
Previously I was on 4.9.63-29 which does not have this problem, and does
have the ldisc patch. So I guess the question is for Johnny, why was it
removed?

In the meantime, I'll revert the kernel and follow up if I see any further
problems.

Cheers,
Nathan

___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-23 Thread Pasi Kärkkäinen 
Hi,

On Tue, Jan 23, 2018 at 10:35:24AM -0800, Nathan March wrote:
> > Thanks for the heads-up.  It's been running through XenServer's tests
> > as well as the XenProject's "osstest" -- I haven't heard of any
> > additional issues, but I'll ask.
> 
> Looks like I can reproduce this pretty easily, this happened upon ssh'ing
> into the server while I had a VM migrating into it. The system goes
> completely unresponsive (can't even enter a keystroke via console):
> 
> [64722.291300] vlan208: port 4(vif5.0) entered forwarding state
> [64722.291695] NOHZ: local_softirq_pending 08
> [64929.006981] BUG: unable to handle kernel paging request at
> 2260
> [64929.007020] IP: [] n_tty_receive_buf_common+0xa4/0x1f0
> [64929.007049] PGD 1f7a53067 [64929.007057] PUD 1ee0d4067 
> PMD 0 [64929.007069] 
> [64929.007077] Oops:  [#1] SMP
> [64929.007088] Modules linked in: ebt_ip6 ebt_ip ebtable_filter ebtables
> arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd auth_rpcgss
> nfsv3 nfs_acl nfs fscache lockd sunrpc grace 8021q mrp garp stp llc bonding
> xen_acpi_processor blktap xen_netback xen_blkback xen_gntdev xen_evtchn
> xenfs xen_privcmd dcdbas fjes pcspkr ipmi_devintf ipmi_si ipmi_msghandler
> joydev i2c_i801 i2c_smbus lpc_ich shpchp mei_me mei ioatdma ixgbe mdio igb
> dca ptp pps_core uas usb_storage wmi ttm
> [64929.007327] CPU: 15 PID: 17696 Comm: kworker/u48:0 Not tainted
> 4.9.75-30.el6.x86_64 #1
> [64929.007343] Hardware name: Dell Inc. PowerEdge C6220/03C9JJ, BIOS 2.7.1
> 03/04/2015
> [64929.007362] Workqueue: events_unbound flush_to_ldisc
> [64929.007376] task: 8801fbc70580 task.stack: c90048af8000
> [64929.007415] RIP: e030:[]  []
> n_tty_receive_buf_common+0xa4/0x1f0
> [64929.007465] RSP: e02b:c90048afbb08  EFLAGS: 00010296
> [64929.007476] RAX: 2260 RBX:  RCX:
> 0002
> [64929.007519] RDX:  RSI: 8801dc0f3c20 RDI:
> 8801f9b8acd8
> [64929.007563] RBP: c90048afbb78 R08: 0001 R09:
> 8210f1c0
> [64929.007577] R10: 7ff0 R11:  R12:
> 0002
> [64929.007620] R13: 8801f9b8ac00 R14:  R15:
> 8801dc0f3c20
> [64929.007675] FS:  7fcfc0af8700() GS:880204dc()
> knlGS:
> [64929.007718] CS:  e033 DS:  ES:  CR0: 80050033
> [64929.007759] CR2: 2260 CR3: 0001f067b000 CR4:
> 00042660
> [64929.007782] Stack:
> [64929.007806]  c90048afbb38  8801f9b8acd8
> 000104dda030
> [64929.007858]  2260 fbc72700 880204dc48c0
> 
> [64929.007941]  880204dce890 8801dc0f3c00 8801f7f25c00
> c90048afbbf8
> [64929.007994] Call Trace:
> [64929.008008]  [] n_tty_receive_buf2+0x14/0x20
> [64929.008048]  [] tty_ldisc_receive_buf+0x23/0x50
>

Hmm.. isn't this the ldisc bug that was discussed a few months ago on this 
list, and a patch was applied to virt-sig kernel aswell? 

Call trace looks similar..


-- Pasi

> [64929.008088]  [] flush_to_ldisc+0xc8/0x100
> [64929.008133]  [] ? __switch_to+0x20b/0x690
> [64929.008176]  [] ? xen_clocksource_read+0x15/0x20
> [64929.008222]  [] process_one_work+0x170/0x500
> [64929.008268]  [] ? __schedule+0x238/0x530
> [64929.008310]  [] ? schedule+0x3a/0xa0
> [64929.008324]  [] worker_thread+0x166/0x530
> [64929.008368]  [] ? put_prev_entity+0x29/0x140
> [64929.008412]  [] ? __schedule+0x238/0x530
> [64929.008458]  [] ? default_wake_function+0x12/0x20
> [64929.008502]  [] ? maybe_create_worker+0x120/0x120
> [64929.008518]  [] ? schedule+0x3a/0xa0
> [64929.008555]  [] ? _raw_spin_unlock_irqrestore+0x16/0x20
> [64929.008599]  [] ? maybe_create_worker+0x120/0x120
> [64929.008616]  [] kthread+0xe5/0x100
> [64929.008630]  [] ? schedule_tail+0x56/0xc0
> [64929.008643]  [] ? __kthread_init_worker+0x40/0x40
> [64929.008659]  [] ? schedule_tail+0x56/0xc0
> [64929.008673]  [] ret_from_fork+0x41/0x50
> [64929.008685] Code: 89 fe 4c 89 ef 89 45 98 e8 aa fb ff ff 8b 45 98 48 63
> d0 48 85 db 48 8d 0c 13 48 0f 45 d9 01 45 bc 49 01 d7 41 29 c4 48 8b 45 b0
> <48> 8b 30 48 89 75 c0 49 8b 0e 8d 96 00 10 00 00 29 ca 41 f6 85 
> [64929.008894] RIP  [] n_tty_receive_buf_common+0xa4/0x1f0
> [64929.008914]  RSP 
> [64929.008923] CR2: 2260
> [64929.009641] ---[ end trace e1da1cdf77fed144 ]---
> [64929.009785] BUG: unable to handle kernel paging request at
> ffd8
> [64929.009804] IP: [] kthread_data+0x10/0x20
> [64929.009823] PGD 200d067 [64929.009831] PUD 200f067 
> PMD 0 [64929.009842] 
> [64929.009850] Oops:  [#2] SMP
> [64929.009864] Modules linked in: ebt_ip6 ebt_ip ebtable_filter ebtables
> arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd auth_rpcgss
> nfsv3 nfs_acl nfs fscache lockd sunrpc grace 8021q mrp garp stp llc bonding
> xen_acpi_processor blktap xen_netback xen_blkback xen_gntdev xen_evtchn
> xenfs xen_privcmd dcdbas fjes pcspkr ipmi_devintf ip

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-23 Thread Nathan March 
> Thanks for the heads-up.  It's been running through XenServer's tests
> as well as the XenProject's "osstest" -- I haven't heard of any
> additional issues, but I'll ask.

Looks like I can reproduce this pretty easily, this happened upon ssh'ing
into the server while I had a VM migrating into it. The system goes
completely unresponsive (can't even enter a keystroke via console):

[64722.291300] vlan208: port 4(vif5.0) entered forwarding state
[64722.291695] NOHZ: local_softirq_pending 08
[64929.006981] BUG: unable to handle kernel paging request at
2260
[64929.007020] IP: [] n_tty_receive_buf_common+0xa4/0x1f0
[64929.007049] PGD 1f7a53067 [64929.007057] PUD 1ee0d4067 
PMD 0 [64929.007069] 
[64929.007077] Oops:  [#1] SMP
[64929.007088] Modules linked in: ebt_ip6 ebt_ip ebtable_filter ebtables
arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd auth_rpcgss
nfsv3 nfs_acl nfs fscache lockd sunrpc grace 8021q mrp garp stp llc bonding
xen_acpi_processor blktap xen_netback xen_blkback xen_gntdev xen_evtchn
xenfs xen_privcmd dcdbas fjes pcspkr ipmi_devintf ipmi_si ipmi_msghandler
joydev i2c_i801 i2c_smbus lpc_ich shpchp mei_me mei ioatdma ixgbe mdio igb
dca ptp pps_core uas usb_storage wmi ttm
[64929.007327] CPU: 15 PID: 17696 Comm: kworker/u48:0 Not tainted
4.9.75-30.el6.x86_64 #1
[64929.007343] Hardware name: Dell Inc. PowerEdge C6220/03C9JJ, BIOS 2.7.1
03/04/2015
[64929.007362] Workqueue: events_unbound flush_to_ldisc
[64929.007376] task: 8801fbc70580 task.stack: c90048af8000
[64929.007415] RIP: e030:[]  []
n_tty_receive_buf_common+0xa4/0x1f0
[64929.007465] RSP: e02b:c90048afbb08  EFLAGS: 00010296
[64929.007476] RAX: 2260 RBX:  RCX:
0002
[64929.007519] RDX:  RSI: 8801dc0f3c20 RDI:
8801f9b8acd8
[64929.007563] RBP: c90048afbb78 R08: 0001 R09:
8210f1c0
[64929.007577] R10: 7ff0 R11:  R12:
0002
[64929.007620] R13: 8801f9b8ac00 R14:  R15:
8801dc0f3c20
[64929.007675] FS:  7fcfc0af8700() GS:880204dc()
knlGS:
[64929.007718] CS:  e033 DS:  ES:  CR0: 80050033
[64929.007759] CR2: 2260 CR3: 0001f067b000 CR4:
00042660
[64929.007782] Stack:
[64929.007806]  c90048afbb38  8801f9b8acd8
000104dda030
[64929.007858]  2260 fbc72700 880204dc48c0

[64929.007941]  880204dce890 8801dc0f3c00 8801f7f25c00
c90048afbbf8
[64929.007994] Call Trace:
[64929.008008]  [] n_tty_receive_buf2+0x14/0x20
[64929.008048]  [] tty_ldisc_receive_buf+0x23/0x50
[64929.008088]  [] flush_to_ldisc+0xc8/0x100
[64929.008133]  [] ? __switch_to+0x20b/0x690
[64929.008176]  [] ? xen_clocksource_read+0x15/0x20
[64929.008222]  [] process_one_work+0x170/0x500
[64929.008268]  [] ? __schedule+0x238/0x530
[64929.008310]  [] ? schedule+0x3a/0xa0
[64929.008324]  [] worker_thread+0x166/0x530
[64929.008368]  [] ? put_prev_entity+0x29/0x140
[64929.008412]  [] ? __schedule+0x238/0x530
[64929.008458]  [] ? default_wake_function+0x12/0x20
[64929.008502]  [] ? maybe_create_worker+0x120/0x120
[64929.008518]  [] ? schedule+0x3a/0xa0
[64929.008555]  [] ? _raw_spin_unlock_irqrestore+0x16/0x20
[64929.008599]  [] ? maybe_create_worker+0x120/0x120
[64929.008616]  [] kthread+0xe5/0x100
[64929.008630]  [] ? schedule_tail+0x56/0xc0
[64929.008643]  [] ? __kthread_init_worker+0x40/0x40
[64929.008659]  [] ? schedule_tail+0x56/0xc0
[64929.008673]  [] ret_from_fork+0x41/0x50
[64929.008685] Code: 89 fe 4c 89 ef 89 45 98 e8 aa fb ff ff 8b 45 98 48 63
d0 48 85 db 48 8d 0c 13 48 0f 45 d9 01 45 bc 49 01 d7 41 29 c4 48 8b 45 b0
<48> 8b 30 48 89 75 c0 49 8b 0e 8d 96 00 10 00 00 29 ca 41 f6 85 
[64929.008894] RIP  [] n_tty_receive_buf_common+0xa4/0x1f0
[64929.008914]  RSP 
[64929.008923] CR2: 2260
[64929.009641] ---[ end trace e1da1cdf77fed144 ]---
[64929.009785] BUG: unable to handle kernel paging request at
ffd8
[64929.009804] IP: [] kthread_data+0x10/0x20
[64929.009823] PGD 200d067 [64929.009831] PUD 200f067 
PMD 0 [64929.009842] 
[64929.009850] Oops:  [#2] SMP
[64929.009864] Modules linked in: ebt_ip6 ebt_ip ebtable_filter ebtables
arptable_filter arp_tables bridge xen_pciback xen_gntalloc nfsd auth_rpcgss
nfsv3 nfs_acl nfs fscache lockd sunrpc grace 8021q mrp garp stp llc bonding
xen_acpi_processor blktap xen_netback xen_blkback xen_gntdev xen_evtchn
xenfs xen_privcmd dcdbas fjes pcspkr ipmi_devintf ipmi_si ipmi_msghandler
joydev i2c_i801 i2c_smbus lpc_ich shpchp mei_me mei ioatdma ixgbe mdio igb
dca ptp pps_core uas usb_storage wmi ttm
[64929.010054] CPU: 15 PID: 17696 Comm: kworker/u48:0 Tainted: G  D
4.9.75-30.el6.x86_64 #1
[64929.010068] Hardware name: Dell Inc. PowerEdge C6220/03C9JJ, BIOS 2.7.1
03/04/2015
[64929.010127] task: 8801fbc70580 task.stack: c90048af8000
[64929.010138] RIP: e030:[]  []
kthread_data

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-23 Thread George Dunlap 
On Mon, Jan 22, 2018 at 10:38 PM, Nathan March  wrote:
> Just a heads up that I'm seeing major stability problems on these builds.
> Didn't have  console capture setup unfortunately, but have seen my test
> hypervisor hard lock twice over the weekend.
>
> This is with xpti being used, rather than the shim.

Thanks for the heads-up.  It's been running through XenServer's tests
as well as the XenProject's "osstest" -- I haven't heard of any
additional issues, but I'll ask.

It's possible also that it's some interaction between the specific
CentOS environment -- the gcc version, the particular dom0 kernel, &c.
I'll ask Anthony if he can run the CentOS packages through osstest and
see if anything comes up.

 -George
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-22 Thread Nathan March 
Just a heads up that I'm seeing major stability problems on these builds.
Didn't have  console capture setup unfortunately, but have seen my test
hypervisor hard lock twice over the weekend.

This is with xpti being used, rather than the shim.

Cheers,
Nathan


> -Original Message-
> From: CentOS-virt [mailto:centos-virt-boun...@centos.org] On Behalf Of
> George Dunlap
> Sent: Wednesday, January 17, 2018 9:14 AM
> To: Discussion about the virtualization on CentOS 
> Subject: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation)
packages
> making their way to centos-virt-xen-testing
> 
> I've built & tagged packages for CentOS 6 and 7 4.6.6-9, with XPTI
> "stage 1" Meltdown mitigation.
> 
> This will allow 64-bit PV guests to run safely (with a few caveats),
> but incurs a fairly significant slowdown for 64-bit PV guests on Intel
> boxes (including domain 0).
> 
> If you prefer using Vixen / Comet, you can turn it off by adding
> 'xpti=0' to your Xen command-line.
> 
> Detailed information can be found in the XSA-254 advisory:
> 
> https://xenbits.xen.org/xsa/advisory-254.html
> 
> Please test and report any issues you have.  I'll probably tag then
> with -release tomorrow.
> 
> 4.8 packages should be coming to buildlogs soon.
> 
>  -George
> ___
> CentOS-virt mailing list
> CentOS-virt@centos.org
> https://lists.centos.org/mailman/listinfo/centos-virt

___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-18 Thread Peter Peltonen 
Hi Nathan,

On Thu, Jan 18, 2018 at 9:31 PM, Nathan March  wrote:
>> -Original Message-
>> As there are now quite many options to choose from, what would be the
>> best option performance wise for running 32bit domUs under xen-4.6?
>>
>> Best,
>> Peter
>>
>
> It's worth taking a look at the table in the latest XSA, it helps clarify a
> fair bit:
>
> https://xenbits.xen.org/xsa/advisory-254.html

thanks for pointing this out, but there is a disclaimer:

"Everything in this section applies to 64-bit PV x86 guests only."

It also reads in the advisory "32-bit PV guests cannot exploit SP3"

So I am wondering if I just "yum update" will I get some fixes I do
not need that will slow my guests down?

Best,
Peter
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-18 Thread Nathan March 
> -Original Message-
> From: CentOS-virt [mailto:centos-virt-boun...@centos.org] On Behalf Of
> Peter Peltonen
> Sent: Thursday, January 18, 2018 11:19 AM
> To: Discussion about the virtualization on CentOS 
> Subject: Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation)
> packages making their way to centos-virt-xen-testing
> 
> Thanks George.
> 
> As there are now quite many options to choose from, what would be the
> best option performance wise for running 32bit domUs under xen-4.6?
> 
> Best,
> Peter
> 

It's worth taking a look at the table in the latest XSA, it helps clarify a
fair bit:

https://xenbits.xen.org/xsa/advisory-254.html

Cheers,
Nathan

___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

Re: [CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-18 Thread Peter Peltonen 
Thanks George.

As there are now quite many options to choose from, what would be the
best option performance wise for running 32bit domUs under xen-4.6?

Best,
Peter


On Wed, Jan 17, 2018 at 7:14 PM, George Dunlap  wrote:
> I've built & tagged packages for CentOS 6 and 7 4.6.6-9, with XPTI
> "stage 1" Meltdown mitigation.
>
> This will allow 64-bit PV guests to run safely (with a few caveats),
> but incurs a fairly significant slowdown for 64-bit PV guests on Intel
> boxes (including domain 0).
>
> If you prefer using Vixen / Comet, you can turn it off by adding
> 'xpti=0' to your Xen command-line.
>
> Detailed information can be found in the XSA-254 advisory:
>
> https://xenbits.xen.org/xsa/advisory-254.html
>
> Please test and report any issues you have.  I'll probably tag then
> with -release tomorrow.
>
> 4.8 packages should be coming to buildlogs soon.
>
>  -George
> ___
> CentOS-virt mailing list
> CentOS-virt@centos.org
> https://lists.centos.org/mailman/listinfo/centos-virt
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt

[CentOS-virt] Xen 4.6.6-9 (with XPTI meltdown mitigation) packages making their way to centos-virt-xen-testing

2018-01-17 Thread George Dunlap 
I've built & tagged packages for CentOS 6 and 7 4.6.6-9, with XPTI
"stage 1" Meltdown mitigation.

This will allow 64-bit PV guests to run safely (with a few caveats),
but incurs a fairly significant slowdown for 64-bit PV guests on Intel
boxes (including domain 0).

If you prefer using Vixen / Comet, you can turn it off by adding
'xpti=0' to your Xen command-line.

Detailed information can be found in the XSA-254 advisory:

https://xenbits.xen.org/xsa/advisory-254.html

Please test and report any issues you have.  I'll probably tag then
with -release tomorrow.

4.8 packages should be coming to buildlogs soon.

 -George
___
CentOS-virt mailing list
CentOS-virt@centos.org
https://lists.centos.org/mailman/listinfo/centos-virt