Private bug reported:
Description of problem:
Observed device_dax related probe error when HBM CPU is set to flat
mode. and creating duplicate device_dax ids and hence probe is failing.
How reproducible:
Frequently
Version-Release
Release: 22.04.2, 22.10
Steps to Reproduce:
1. Set HBM cpu to flat mode in memory settings in BIOS.
2. Boot to the OS.
3. Perform OS warm boot cycle test.
4. Observe the dax2.0/dax3.0/dax4.0/dax5.0 probe error.
Actual results:
Observed device_dax related errors in dmesg, device Dax is creating
dummy/duplicate devices and probe failing.
Expected results:
Demmy/duplicate devices should not create.
Additional info:
SUT is having 2*32C HBM cpus. Eligible system-ram mode change devices should be
only 2[dax0.0, dax1.0], but under "daxctl list -u" is showing 1st time 4
devices [dax0.0, 1.0, 2.0, 3.0], 2 is "state":"disabled" and 2 more devices is
"mode":"devdax" which are actuall devadax to system-ram convertible devices.
After reconfigure-device dax0.0, dax1.0 when you list the devices couple of
more dummy/dumplicate devices are creating with "state":"disabled"[Ex: dax4.0,
5.0 etc..].
root@ubuntu:/home/ubuntu# daxctl list -u
[
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"devdax"---------------> HBM CPU 1, This we can change the devdax to
system-ram
},
{
"chardev":"dax2.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax3.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"devdax" ---------------> HBM CPU 1, This we can change the devdax to
system-ram
}
]
root@ubuntu:/home/ubuntu# dmesg | grep -i error
[ 12.748884] device_dax: probe of dax2.0 failed with error -16
[ 12.748902] device_dax: probe of dax3.0 failed with error -16
After reconfig-device devdax to system-ram below are the results:
-------------------------------------------------------------------
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax0.0 -u
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"system-ram",
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax1.0 -u
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"system-ram",
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl list -u
[
{
"chardev":"dax4.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"system-ram",-----------> Converted from devdax - system-ram
"online_memblocks":32,
"total_memblocks":32,
"movable":true
},
{
"chardev":"dax5.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax2.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax3.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"system-ram", -----------> Converted from devdax - system-ram
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
]
root@ubuntu:/home/ubuntu# dmesg | grep -i dax
[ 12.748880] device_dax dax2.0: mapping0: 0x2080000000-0x307fffffff could not
reserve range
[ 12.748884] device_dax: probe of dax2.0 failed with error -16
[ 12.748901] device_dax dax3.0: mapping0: 0x5080000000-0x607fffffff could not
reserve range
[ 12.748902] device_dax: probe of dax3.0 failed with error -16
[ 812.677056] device_dax dax4.0: mapping0: 0x2080000000-0x307fffffff could not
reserve range
[ 812.677070] device_dax: probe of dax4.0 failed with error -16
[ 821.092762] device_dax dax5.0: mapping0: 0x5080000000-0x607fffffff could not
reserve range
[ 821.092774] device_dax: probe of dax5.0 failed with error -16
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Information type changed from Public to Private
** Description changed:
Description of problem:
Observed device_dax related probe error when HBM CPU is set to flat
mode. and creating duplicate device_dax ids and hence probe is failing.
How reproducible:
Frequently
+
+ Version-Release
+ Release: 22.04.2, 22.10
+
Steps to Reproduce:
1. Set HBM cpu to flat mode in memory settings in BIOS.
2. Boot to the OS.
3. Perform OS warm boot cycle test.
4. Observe the dax2.0/dax3.0/dax4.0/dax5.0 probe error.
Actual results:
Observed device_dax related errors in dmesg, device Dax is creating
dummy/duplicate devices and probe failing.
Expected results:
Demmy/duplicate devices should not create.
Additional info:
SUT is having 2*32C HBM cpus. Eligible system-ram mode change devices should
be only 2[dax0.0, dax1.0], but under "daxctl list -u" is showing 1st time 4
devices [dax0.0, 1.0, 2.0, 3.0], 2 is "state":"disabled" and 2 more devices is
"mode":"devdax" which are actuall devadax to system-ram convertible devices.
After reconfigure-device dax0.0, dax1.0 when you list the devices couple of
more dummy/dumplicate devices are creating with "state":"disabled"[Ex: dax4.0,
5.0 etc..].
-
root@ubuntu:/home/ubuntu# daxctl list -u
[
- {
- "chardev":"dax1.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":3,
- "align":2097152,
- "mode":"devdax"---------------> HBM CPU 1, This we can change the devdax
to
- system-ram
- },
- {
- "chardev":"dax2.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":2, --------------------> Duplicate device
- "align":2097152,
- "mode":"devdax",
- "state":"disabled"
- },
- {
- "chardev":"dax3.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":3, --------------------> Duplicate device
- "align":2097152,
- "mode":"devdax",
- "state":"disabled"
- },
- {
- "chardev":"dax0.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":2,
- "align":2097152,
- "mode":"devdax" ---------------> HBM CPU 1, This we can change the devdax
to
- system-ram
- }
+ {
+ "chardev":"dax1.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":3,
+ "align":2097152,
+ "mode":"devdax"---------------> HBM CPU 1, This we can change the devdax
to
+ system-ram
+ },
+ {
+ "chardev":"dax2.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":2, --------------------> Duplicate device
+ "align":2097152,
+ "mode":"devdax",
+ "state":"disabled"
+ },
+ {
+ "chardev":"dax3.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":3, --------------------> Duplicate device
+ "align":2097152,
+ "mode":"devdax",
+ "state":"disabled"
+ },
+ {
+ "chardev":"dax0.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":2,
+ "align":2097152,
+ "mode":"devdax" ---------------> HBM CPU 1, This we can change the devdax
to
+ system-ram
+ }
]
root@ubuntu:/home/ubuntu# dmesg | grep -i error
[ 12.748884] device_dax: probe of dax2.0 failed with error -16
[ 12.748902] device_dax: probe of dax3.0 failed with error -16
-
After reconfig-device devdax to system-ram below are the results:
-------------------------------------------------------------------
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax0.0 -u
{
- "chardev":"dax0.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":2,
- "align":2097152,
- "mode":"system-ram",
- "online_memblocks":32,
- "total_memblocks":32,
- "movable":true
+ "chardev":"dax0.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":2,
+ "align":2097152,
+ "mode":"system-ram",
+ "online_memblocks":32,
+ "total_memblocks":32,
+ "movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax1.0 -u
{
- "chardev":"dax1.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":3,
- "align":2097152,
- "mode":"system-ram",
- "online_memblocks":32,
- "total_memblocks":32,
- "movable":true
+ "chardev":"dax1.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":3,
+ "align":2097152,
+ "mode":"system-ram",
+ "online_memblocks":32,
+ "total_memblocks":32,
+ "movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl list -u
[
- {
- "chardev":"dax4.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":2, --------------------> Duplicate device
- "align":2097152,
- "mode":"devdax",
- "state":"disabled"
- },
- {
- "chardev":"dax1.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":3,
- "align":2097152,
- "mode":"system-ram",-----------> Converted from devdax - system-ram
- "online_memblocks":32,
- "total_memblocks":32,
- "movable":true
- },
- {
- "chardev":"dax5.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":3, --------------------> Duplicate device
- "align":2097152,
- "mode":"devdax",
- "state":"disabled"
- },
- {
- "chardev":"dax2.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":2, --------------------> Duplicate device
- "align":2097152,
- "mode":"devdax",
- "state":"disabled"
- },
- {
- "chardev":"dax3.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":3, --------------------> Duplicate device
- "align":2097152,
- "mode":"devdax",
- "state":"disabled"
- },
- {
- "chardev":"dax0.0",
- "size":"64.00 GiB (68.72 GB)",
- "target_node":2,
- "align":2097152,
- "mode":"system-ram", -----------> Converted from devdax - system-ram
- "online_memblocks":32,
- "total_memblocks":32,
- "movable":true
- }
+ {
+ "chardev":"dax4.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":2, --------------------> Duplicate device
+ "align":2097152,
+ "mode":"devdax",
+ "state":"disabled"
+ },
+ {
+ "chardev":"dax1.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":3,
+ "align":2097152,
+ "mode":"system-ram",-----------> Converted from devdax - system-ram
+ "online_memblocks":32,
+ "total_memblocks":32,
+ "movable":true
+ },
+ {
+ "chardev":"dax5.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":3, --------------------> Duplicate device
+ "align":2097152,
+ "mode":"devdax",
+ "state":"disabled"
+ },
+ {
+ "chardev":"dax2.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":2, --------------------> Duplicate device
+ "align":2097152,
+ "mode":"devdax",
+ "state":"disabled"
+ },
+ {
+ "chardev":"dax3.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":3, --------------------> Duplicate device
+ "align":2097152,
+ "mode":"devdax",
+ "state":"disabled"
+ },
+ {
+ "chardev":"dax0.0",
+ "size":"64.00 GiB (68.72 GB)",
+ "target_node":2,
+ "align":2097152,
+ "mode":"system-ram", -----------> Converted from devdax - system-ram
+ "online_memblocks":32,
+ "total_memblocks":32,
+ "movable":true
+ }
]
** Description changed:
Description of problem:
Observed device_dax related probe error when HBM CPU is set to flat
mode. and creating duplicate device_dax ids and hence probe is failing.
How reproducible:
Frequently
Version-Release
Release: 22.04.2, 22.10
-
Steps to Reproduce:
1. Set HBM cpu to flat mode in memory settings in BIOS.
2. Boot to the OS.
3. Perform OS warm boot cycle test.
4. Observe the dax2.0/dax3.0/dax4.0/dax5.0 probe error.
Actual results:
Observed device_dax related errors in dmesg, device Dax is creating
dummy/duplicate devices and probe failing.
Expected results:
Demmy/duplicate devices should not create.
Additional info:
SUT is having 2*32C HBM cpus. Eligible system-ram mode change devices should
be only 2[dax0.0, dax1.0], but under "daxctl list -u" is showing 1st time 4
devices [dax0.0, 1.0, 2.0, 3.0], 2 is "state":"disabled" and 2 more devices is
"mode":"devdax" which are actuall devadax to system-ram convertible devices.
After reconfigure-device dax0.0, dax1.0 when you list the devices couple of
more dummy/dumplicate devices are creating with "state":"disabled"[Ex: dax4.0,
5.0 etc..].
root@ubuntu:/home/ubuntu# daxctl list -u
[
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"devdax"---------------> HBM CPU 1, This we can change the devdax
to
system-ram
},
{
"chardev":"dax2.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax3.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"devdax" ---------------> HBM CPU 1, This we can change the devdax
to
system-ram
}
]
root@ubuntu:/home/ubuntu# dmesg | grep -i error
[ 12.748884] device_dax: probe of dax2.0 failed with error -16
[ 12.748902] device_dax: probe of dax3.0 failed with error -16
After reconfig-device devdax to system-ram below are the results:
-------------------------------------------------------------------
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax0.0 -u
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"system-ram",
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax1.0 -u
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"system-ram",
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl list -u
[
{
"chardev":"dax4.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"system-ram",-----------> Converted from devdax - system-ram
"online_memblocks":32,
"total_memblocks":32,
"movable":true
},
{
"chardev":"dax5.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax2.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax3.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"system-ram", -----------> Converted from devdax - system-ram
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
]
+
+ root@ubuntu:/home/ubuntu# dmesg | grep -i dax
+ [ 12.748880] device_dax dax2.0: mapping0: 0x2080000000-0x307fffffff could
not reserve range
+ [ 12.748884] device_dax: probe of dax2.0 failed with error -16
+ [ 12.748901] device_dax dax3.0: mapping0: 0x5080000000-0x607fffffff could
not reserve range
+ [ 12.748902] device_dax: probe of dax3.0 failed with error -16
+ [ 812.677056] device_dax dax4.0: mapping0: 0x2080000000-0x307fffffff could
not reserve range
+ [ 812.677070] device_dax: probe of dax4.0 failed with error -16
+ [ 821.092762] device_dax dax5.0: mapping0: 0x5080000000-0x607fffffff could
not reserve range
+ [ 821.092774] device_dax: probe of dax5.0 failed with error -16
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2028158
Title:
Observed device_dax related probe errors in dmesg when HBM CPU is set
to flat mode and creating duplicate device_dax ids and hence probe is
failing.
Status in linux package in Ubuntu:
New
Bug description:
Description of problem:
Observed device_dax related probe error when HBM CPU is set to flat
mode. and creating duplicate device_dax ids and hence probe is
failing.
How reproducible:
Frequently
Version-Release
Release: 22.04.2, 22.10
Steps to Reproduce:
1. Set HBM cpu to flat mode in memory settings in BIOS.
2. Boot to the OS.
3. Perform OS warm boot cycle test.
4. Observe the dax2.0/dax3.0/dax4.0/dax5.0 probe error.
Actual results:
Observed device_dax related errors in dmesg, device Dax is creating
dummy/duplicate devices and probe failing.
Expected results:
Demmy/duplicate devices should not create.
Additional info:
SUT is having 2*32C HBM cpus. Eligible system-ram mode change devices should
be only 2[dax0.0, dax1.0], but under "daxctl list -u" is showing 1st time 4
devices [dax0.0, 1.0, 2.0, 3.0], 2 is "state":"disabled" and 2 more devices is
"mode":"devdax" which are actuall devadax to system-ram convertible devices.
After reconfigure-device dax0.0, dax1.0 when you list the devices couple of
more dummy/dumplicate devices are creating with "state":"disabled"[Ex: dax4.0,
5.0 etc..].
root@ubuntu:/home/ubuntu# daxctl list -u
[
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"devdax"---------------> HBM CPU 1, This we can change the devdax
to
system-ram
},
{
"chardev":"dax2.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax3.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"devdax" ---------------> HBM CPU 1, This we can change the devdax
to
system-ram
}
]
root@ubuntu:/home/ubuntu# dmesg | grep -i error
[ 12.748884] device_dax: probe of dax2.0 failed with error -16
[ 12.748902] device_dax: probe of dax3.0 failed with error -16
After reconfig-device devdax to system-ram below are the results:
-------------------------------------------------------------------
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax0.0 -u
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"system-ram",
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl reconfigure-device -m system-ram dax1.0 -u
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"system-ram",
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
reconfigured 1 device
root@ubuntu:/home/ubuntu# daxctl list -u
[
{
"chardev":"dax4.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax1.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3,
"align":2097152,
"mode":"system-ram",-----------> Converted from devdax - system-ram
"online_memblocks":32,
"total_memblocks":32,
"movable":true
},
{
"chardev":"dax5.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax2.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax3.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":3, --------------------> Duplicate device
"align":2097152,
"mode":"devdax",
"state":"disabled"
},
{
"chardev":"dax0.0",
"size":"64.00 GiB (68.72 GB)",
"target_node":2,
"align":2097152,
"mode":"system-ram", -----------> Converted from devdax - system-ram
"online_memblocks":32,
"total_memblocks":32,
"movable":true
}
]
root@ubuntu:/home/ubuntu# dmesg | grep -i dax
[ 12.748880] device_dax dax2.0: mapping0: 0x2080000000-0x307fffffff could
not reserve range
[ 12.748884] device_dax: probe of dax2.0 failed with error -16
[ 12.748901] device_dax dax3.0: mapping0: 0x5080000000-0x607fffffff could
not reserve range
[ 12.748902] device_dax: probe of dax3.0 failed with error -16
[ 812.677056] device_dax dax4.0: mapping0: 0x2080000000-0x307fffffff could
not reserve range
[ 812.677070] device_dax: probe of dax4.0 failed with error -16
[ 821.092762] device_dax dax5.0: mapping0: 0x5080000000-0x607fffffff could
not reserve range
[ 821.092774] device_dax: probe of dax5.0 failed with error -16
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2028158/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
