Re: modload & xen and -current 9.99.60
A short summary from my private discussion with Manuel: - the tripping point in my setup was around 44096 Kb - type="pvh" works in large memory constellations - Thus is issue seems to exist primarily in DOMU kernels So it is similar, but not the same. In my scenario modloads fail when a certain amount or more of memory is available. I switched to type=pvh new. Frank On 05/11/20 15:15, Stephen Borrill wrote: On Fri, 8 May 2020, Manuel Bouyer wrote: On Fri, May 08, 2020 at 02:55:10PM +0200, Frank Kardel wrote: I checked to same kernel in an instance with memory=2048 and it just works. Using todays kernel also works woth memory=2048. Using memory=65536 for the xen instance gives a surprising familiar TEST-A# modload bpfjit [ 97.4727034] kobj_load, 444: [%M/bpfjit/bpfjit.kmod]: linker error: out of memory modload: bpfjit: Cannot allocate memory TEST-A# So it seems to be linked to available memory. The more you have the less you get for modload. It could be a variable overflow somewhere but I can't see how it relates to 64Gb. Does it work with 16Gb ? This sounds similar to the problem I reported a couple of weeks ago with exactly 16GB: http://mail-index.netbsd.org/port-xen/2020/04/17/msg009654.html Also could you try with a PVH or HVM guest ? These ones would use modules from /stand/amd64/ and not /stand/amd64-xen/ and should be close to native. I don't have a box with that much RAM to test ... -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: modload & xen and -current 9.99.60
On Fri, 8 May 2020, Manuel Bouyer wrote: On Fri, May 08, 2020 at 02:55:10PM +0200, Frank Kardel wrote: I checked to same kernel in an instance with memory=2048 and it just works. Using todays kernel also works woth memory=2048. Using memory=65536 for the xen instance gives a surprising familiar TEST-A# modload bpfjit [ 97.4727034] kobj_load, 444: [%M/bpfjit/bpfjit.kmod]: linker error: out of memory modload: bpfjit: Cannot allocate memory TEST-A# So it seems to be linked to available memory. The more you have the less you get for modload. It could be a variable overflow somewhere but I can't see how it relates to 64Gb. Does it work with 16Gb ? This sounds similar to the problem I reported a couple of weeks ago with exactly 16GB: http://mail-index.netbsd.org/port-xen/2020/04/17/msg009654.html Also could you try with a PVH or HVM guest ? These ones would use modules from /stand/amd64/ and not /stand/amd64-xen/ and should be close to native. I don't have a box with that much RAM to test ... -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: modload & xen and -current 9.99.60
On Fri, May 08, 2020 at 02:55:10PM +0200, Frank Kardel wrote: > I checked to same kernel in an instance with memory=2048 and it just works. > > Using todays kernel also works woth memory=2048. > > Using memory=65536 for the xen instance gives a surprising familiar > > TEST-A# modload bpfjit > [ 97.4727034] kobj_load, 444: [%M/bpfjit/bpfjit.kmod]: linker error: out of > memory > modload: bpfjit: Cannot allocate memory > TEST-A# > > So it seems to be linked to available memory. > > The more you have the less you get for modload. It could be a variable overflow somewhere but I can't see how it relates to 64Gb. Does it work with 16Gb ? Also could you try with a PVH or HVM guest ? These ones would use modules from /stand/amd64/ and not /stand/amd64-xen/ and should be close to native. I don't have a box with that much RAM to test ... -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: modload & xen and -current 9.99.60
I checked to same kernel in an instance with memory=2048 and it just works. Using todays kernel also works woth memory=2048. Using memory=65536 for the xen instance gives a surprising familiar TEST-A# modload bpfjit [ 97.4727034] kobj_load, 444: [%M/bpfjit/bpfjit.kmod]: linker error: out of memory modload: bpfjit: Cannot allocate memory TEST-A# So it seems to be linked to available memory. The more you have the less you get for modload. Frank On 05/07/20 22:52, Manuel Bouyer wrote: On Thu, May 07, 2020 at 09:50:18PM +0200, Frank Kardel wrote: see here: Alpine: 21:45 ~ [8] sysctl kern.module.path kern.module.path = /stand/amd64-xen/9.99.60/modules looks good Alpine: 21:46 ~ [9] ll /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod -r--r--r-- 1 root wheel 34328 May 5 16:58 /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod Alpine: 21:46 ~ [10] size /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod textdata bss dec hex filename 10399 0 0 10399289f /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod Alpine: 21:46 ~ [11] ll /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod -r--r--r-- 1 root wheel 140600 May 5 16:55 /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod Alpine: 21:47 ~ [12] size /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod textdata bss dec hex filename 132575 16 0 132591 205ef /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod no problem for me, with sources from today: xen1:/#modload bpfjit xen1:/#modstat | grep !$รถ modstat | grep bpfjit bpfjit misc filesys -09174 sljit xen1:/#modload pciverbose xen1:/#modstat | grep !$ modstat | grep pciverbose pciverbose misc filesys -0 218 pci
Re: modload & xen and -current 9.99.60
Will test again with a newer kernel. I forgot to mention the that DOMU is configured with memory=40. I will also test with a lower memory value. Frank On 05/07/20 22:52, Manuel Bouyer wrote: On Thu, May 07, 2020 at 09:50:18PM +0200, Frank Kardel wrote: see here: Alpine: 21:45 ~ [8] sysctl kern.module.path kern.module.path = /stand/amd64-xen/9.99.60/modules looks good Alpine: 21:46 ~ [9] ll /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod -r--r--r-- 1 root wheel 34328 May 5 16:58 /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod Alpine: 21:46 ~ [10] size /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod textdata bss dec hex filename 10399 0 0 10399289f /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod Alpine: 21:46 ~ [11] ll /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod -r--r--r-- 1 root wheel 140600 May 5 16:55 /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod Alpine: 21:47 ~ [12] size /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod textdata bss dec hex filename 132575 16 0 132591 205ef /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod no problem for me, with sources from today: xen1:/#modload bpfjit xen1:/#modstat | grep !$ modstat | grep bpfjit bpfjit misc filesys -09174 sljit xen1:/#modload pciverbose xen1:/#modstat | grep !$ modstat | grep pciverbose pciverbose misc filesys -0 218 pci
Re: modload & xen and -current 9.99.60
On Thu, May 07, 2020 at 09:50:18PM +0200, Frank Kardel wrote: > see here: > > Alpine: 21:45 ~ [8] sysctl kern.module.path > kern.module.path = /stand/amd64-xen/9.99.60/modules looks good > Alpine: 21:46 ~ [9] ll /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod > -r--r--r-- 1 root wheel 34328 May 5 16:58 > /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod > Alpine: 21:46 ~ [10] size > /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod >textdata bss dec hex filename > 10399 0 0 10399289f > /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod > Alpine: 21:46 ~ [11] ll > /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod > -r--r--r-- 1 root wheel 140600 May 5 16:55 > /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod > Alpine: 21:47 ~ [12] size > /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod >textdata bss dec hex filename > 132575 16 0 132591 205ef > /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod no problem for me, with sources from today: xen1:/#modload bpfjit xen1:/#modstat | grep !$ modstat | grep bpfjit bpfjit misc filesys -09174 sljit xen1:/#modload pciverbose xen1:/#modstat | grep !$ modstat | grep pciverbose pciverbose misc filesys -0 218 pci -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
Re: modload & xen and -current 9.99.60
see here: Alpine: 21:45 ~ [8] sysctl kern.module.path kern.module.path = /stand/amd64-xen/9.99.60/modules Alpine: 21:46 ~ [9] ll /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod -r--r--r-- 1 root wheel 34328 May 5 16:58 /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod Alpine: 21:46 ~ [10] size /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod textdata bss dec hex filename 10399 0 0 10399289f /stand/amd64-xen/9.99.60/modules/bpfjit/bpfjit.kmod Alpine: 21:46 ~ [11] ll /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod -r--r--r-- 1 root wheel 140600 May 5 16:55 /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod Alpine: 21:47 ~ [12] size /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod textdata bss dec hex filename 132575 16 0 132591 205ef /stand/amd64-xen/9.99.60/modules/pciverbose/pciverbose.kmod Frank On 05/07/20 17:40, Manuel Bouyer wrote: On Thu, May 07, 2020 at 07:45:48AM +0200, Frank Kardel wrote: Hi, Running 9.99.60 XEN3_DOMU shows [ 67264.313173] kobj_load, 444: [%M/bpfjit/bpfjit.kmod]: linker error: out of memory [ 67292.894143] kobj_load, 428: [%M/scsiverbose/scsiverbose.kmod]: linker error: out of memory and modload fails with the OOM error. Is this an expected behavior or a bug? (kern.securelevel is -1). What does kern.module.path show for you ?
Re: modload & xen and -current 9.99.60
On Thu, May 07, 2020 at 07:45:48AM +0200, Frank Kardel wrote: > Hi, > > Running 9.99.60 XEN3_DOMU shows > > [ 67264.313173] kobj_load, 444: [%M/bpfjit/bpfjit.kmod]: linker error: out > of memory > [ 67292.894143] kobj_load, 428: [%M/scsiverbose/scsiverbose.kmod]: linker > error: out of memory > > and modload fails with the OOM error. > > Is this an expected behavior or a bug? (kern.securelevel is -1). What does kern.module.path show for you ? -- Manuel Bouyer NetBSD: 26 ans d'experience feront toujours la difference --
modload & xen and -current 9.99.60
Hi, Running 9.99.60 XEN3_DOMU shows [ 67264.313173] kobj_load, 444: [%M/bpfjit/bpfjit.kmod]: linker error: out of memory [ 67292.894143] kobj_load, 428: [%M/scsiverbose/scsiverbose.kmod]: linker error: out of memory and modload fails with the OOM error. Is this an expected behavior or a bug? (kern.securelevel is -1). Frank
