[Bug 1973758] Re: Azure: Mellanox VF NIC crashes when removed
I checked with Matthew and found Matthew only applied the first patch [1]; after I applied the second patch [2], I'm no longer seeing any crash or memory corruption issue in Matthew's VM. BTW, the Windows Server 2019 host running Matthew's VM doesn't work with NIC SR-IOV correctly: when SR-IOV is enabled, the host offers an Intel VF NIC to the VM, then immediately removes/rescinds the VF (this causes hv_pci_probe() to fail and the bug on its error handling path is triggered), and never re-offers the VF, i.e. NIC SR-IOV doesn't work on this host, but that's a host bug and the host team needs to investigate that. [0] https://lists.ubuntu.com/archives/kernel-team/2022-May/130378.html [1] https://lists.ubuntu.com/archives/kernel-team/2022-May/130379.html [2] https://lists.ubuntu.com/archives/kernel-team/2022-May/130380.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1973758 Title: Azure: Mellanox VF NIC crashes when removed To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1973758/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1965618] Re: linux-azure: Focal 5.4 arm64 support
The 5.4.0-1075-azure and newer kernels are broken in that the VM can easily panic when the Mellanox VF NIC is removed and added due to Azure host servicing events or the below manual "unbind/bind" test (here the GUID can be different in different VMs): for i in `seq 1 1000`; do cd /sys/bus/vmbus/drivers/hv_pci; echo abdc2107-402e-4704-8c88-c2b850696c3c > unbind; echo abdc2107-402e-4704-8c88-c2b850696c3c > bind; done A sample panic call-trace is: [ 107.359954] kernel BUG at /build/linux-azure-5.4-4I3kFs/linux-azure-5.4-5.4.0/mm/slub.c:4020! [ 107.363858] invalid opcode: [#1] SMP NOPTI [ 107.365870] CPU: 0 PID: 334 Comm: kworker/0:2 Not tainted 5.4.0-1077-azure #80~18.04.1-Ubuntu [ 107.369589] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018 [ 107.373811] Workqueue: events vmbus_onmessage_work [ 107.375909] RIP: 0010:kfree+0x1d2/0x240 … [ 107.413789] Call Trace: [ 107.414867] kobject_uevent_env+0x1b5/0x7e0 [ 107.416747] kobject_uevent+0xb/0x10 [ 107.418327] device_release_driver_internal+0x191/0x1c0 [ 107.420653] device_release_driver+0x12/0x20 [ 107.422523] bus_remove_device+0xe1/0x150 [ 107.424279] device_del+0x167/0x380 [ 107.425824] device_unregister+0x1a/0x60 [ 107.427536] vmbus_device_unregister+0x27/0x50 [ 107.429528] vmbus_onoffer_rescind+0x1d0/0x1f0 [ 107.431474] vmbus_onmessage+0x2c/0x70 [ 107.433104] vmbus_onmessage_work+0x22/0x30 [ 107.434919] process_one_work+0x209/0x400 [ 107.436661] worker_thread+0x34/0x40 It turns out there is a bug in https://git.launchpad.net/~canonical- kernel/ubuntu/+source/linux-azure/+git/bionic/commit/?id=16a3c750a78d8, which misses the second hunk of the upstream patch https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=877b911a5ba0. Please apply the below patch to fix the issue: --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -3653,7 +3653,7 @@ static int hv_pci_remove(struct hv_device *hdev) hv_put_dom_num(hbus->bridge->domain_nr); - free_page((unsigned long)hbus); + kfree(hbus); return ret; } BTW, please apply this patch as well (Note: this patch is not really required as it's only for error handling path, which is usually unlikely): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=42c3d41832ef4fcf60aaa6f748de01ad99572adf -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1965618 Title: linux-azure: Focal 5.4 arm64 support To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1965618/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1959216] Re: linux-azure: CONFIG_FB_EFI=y
Got it. I apprecite the clarification. Obviously I was not clear on this :-) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1959216 Title: linux-azure: CONFIG_FB_EFI=y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1959216/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1959216] Re: linux-azure: CONFIG_FB_EFI=y
Hi Tim, thanks! I saw the CONFIG_FB_EFI=y change to the v5.4 linux-azure focal/master-next branch here: https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-azure/+git/focal/log/?h=master-next My Ubuntu 18.04 (Bionic) VM, which was created from the Azure marketplace Ubuntu 18.04 image, has the kernel 5.4 kernel by default: root@decui-u1804-cifs-101:~# cat /etc/os-release NAME="Ubuntu" VERSION="18.04.6 LTS (Bionic Beaver)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 18.04.6 LTS" VERSION_ID="18.04" HOME_URL="https://www.ubuntu.com/; SUPPORT_URL="https://help.ubuntu.com/; BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/; PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy; VERSION_CODENAME=bionic UBUNTU_CODENAME=bionic root@decui-u1804-cifs-101:~# uname -a Linux decui-u1804-cifs-101 5.4.0-1068-azure #71~18.04.1-Ubuntu SMP Thu Jan 20 08:21:40 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux I suppose my 18.04 VM will automatically get a new 5.4 kernel that has CONFIG_FB_EFI=y. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1959216 Title: linux-azure: CONFIG_FB_EFI=y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1959216/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1959216] Re: linux-azure: CONFIG_FB_EFI=y
typo... By "but the 5.14-based one is not listed.", I meant the "... 5.4-based ...", not 5.14-based. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1959216 Title: linux-azure: CONFIG_FB_EFI=y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1959216/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1959216] Re: linux-azure: CONFIG_FB_EFI=y
The 5.4-based linux-azure kenel (https://git.launchpad.net/~canonical- kernel/ubuntu/+source/linux-azure/+git/bionic/log/?h=azure-5.4) is also affected by the bug. Will it be fixed as well? I see the 4.15, 5.11, 5.13 linux-azure kernels are explicited listed at the top of this page, but the 5.14-based one is not listed. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1959216 Title: linux-azure: CONFIG_FB_EFI=y To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1959216/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1928788] Re: linux-azure: Add Mana network driver
I installed and tested the 5.8.0-1034-azure kernel and it worked as expected. I created a Ubuntu 20.04 VM and installed the “5.8.0-1034” kernel this way: 1. Enable the “proposed” kernel by running the below as “root” (refer to https://wiki.ubuntu.com/Testing/EnableProposed): cat
[Bug 1928269] Re: netfilter: iptables-restore: setsockopt(3, SOL_IP, IPT_SO_SET_REPLACE, "security...", ...) return -EAGAIN
I reported the issue to the mailing list: https://lwn.net/ml/linux-kernel/MW2PR2101MB0892FC0F67BD25661CDCE149BF529%40MW2PR2101MB0892.namprd21.prod.outlook.com/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1928269 Title: netfilter: iptables-restore: setsockopt(3, SOL_IP, IPT_SO_SET_REPLACE, "security...", ...) return -EAGAIN To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1928269/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1928269] [NEW] netfilter: iptables-restore: setsockopt(3, SOL_IP, IPT_SO_SET_REPLACE, "security...", ...) return -EAGAIN
Public bug reported:
Hi,
I'm debugging an iptables-restore failure, which happens about 5% of the
time when I keep stopping and starting the Linux VM. The VM has only 1
CPU, and kernel version is 4.15.0-1098-azure, but I suspect the issue may
also exist in the mainline Linux kernel.
When the failure happens, it's always caused by line 27 of the rule
file:
1 # Generated by iptables-save v1.6.0 on Fri Apr 23 09:22:59 2021
2 *raw
3 :PREROUTING ACCEPT [0:0]
4 :OUTPUT ACCEPT [0:0]
5 -A PREROUTING ! -s 168.63.129.16/32 -p tcp -j NOTRACK
6 -A OUTPUT ! -d 168.63.129.16/32 -p tcp -j NOTRACK
7 COMMIT
8 # Completed on Fri Apr 23 09:22:59 2021
9 # Generated by iptables-save v1.6.0 on Fri Apr 23 09:22:59 2021
10 *filter
11 :INPUT ACCEPT [2407:79190058]
12 :FORWARD ACCEPT [0:0]
13 :OUTPUT ACCEPT [1648:2190051]
14 -A OUTPUT -d 169.254.169.254/32 -m owner --uid-owner 33 -j DROP
15 COMMIT
16 # Completed on Fri Apr 23 09:22:59 2021
17 # Generated by iptables-save v1.6.0 on Fri Apr 23 09:22:59 2021
18 *security
19 :INPUT ACCEPT [2345:79155398]
20 :FORWARD ACCEPT [0:0]
21 :OUTPUT ACCEPT [1504:2129015]
22 -A OUTPUT -d 168.63.129.16/32 -p tcp -m owner --uid-owner 0 -j ACCEPT
23 -A OUTPUT -d 168.63.129.16/32 -p tcp -m conntrack --ctstate INVALID,NEW -j
DROP
24 -A OUTPUT -d 168.63.129.16/32 -p tcp -m owner --uid-owner 0 -j ACCEPT
25 -A OUTPUT -d 168.63.129.16/32 -p tcp -m conntrack --ctstate INVALID,NEW -j
DROP
26 -A OUTPUT -d 168.63.129.16/32 -p tcp -m conntrack --ctstate INVALID,NEW -j
DROP
27 COMMIT
The related part of the strace log is:
1 socket(PF_INET, SOCK_RAW, IPPROTO_RAW) = 3
2 getsockopt(3, SOL_IP, IPT_SO_GET_INFO,
"security\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., [84]) = 0
3 getsockopt(3, SOL_IP, IPT_SO_GET_ENTRIES,
"security\0\357B\16Z\177\0\0Pg\355\0\0\0\0\0Pg\355\0\0\0\0\0"..., [880]) = 0
4 setsockopt(3, SOL_IP, IPT_SO_SET_REPLACE,
"security\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2200) = -1
EAGAIN (Resource temporarily unavailable)
5 close(3) = 0
6 write(2, "iptables-restore: line 27 failed"..., 33) = 33
The -EAGAIN error comes from line 1240 of xt_replace_table():
do_ipt_set_ctl
do_replace
__do_replace
xt_replace_table
1216 xt_replace_table(struct xt_table *table,
1217 unsigned int num_counters,
1218 struct xt_table_info *newinfo,
1219 int *error)
1220 {
1221 struct xt_table_info *private;
1222 unsigned int cpu;
1223 int ret;
1224
1225 ret = xt_jumpstack_alloc(newinfo);
1226 if (ret < 0) {
1227 *error = ret;
1228 return NULL;
1229 }
1230
1231 /* Do the substitution. */
1232 local_bh_disable();
1233 private = table->private;
1234
1235 /* Check inside lock: is the old number correct? */
1236 if (num_counters != private->number) {
1237 pr_debug("num_counters != table->private->number
(%u/%u)\n",
1238 num_counters, private->number);
1239 local_bh_enable();
1240 *error = -EAGAIN;
1241 return NULL;
1242 }
When the function returns -EAGAIN, the 'num_counters' is 5 while
'private->number' is 6.
If I re-run the iptables-restore program upon the failure, the program
will succeed.
I checked the function xt_replace_table() in the recent mainline kernel and it
looks like the function is the same.
It looks like there is a race condition between iptables-restore calls
getsockopt() to get the number of table entries and iptables call
setsockopt() to replace the entries? Looks like some other program is
concurrently calling getsockopt()/setsockopt() -- but it looks like this is
not the case according to the messages I print via trace_printk() around
do_replace() in do_ipt_set_ctl(): when the -EAGAIN error happens, there is
no other program calling do_replace(); the table entry number was changed
to 5 by another program 'iptables' about 1.3 milliseconds ago, and then
this program 'iptables-restore' calls setsockopt() and the kernel sees
'num_counters' being 5 and the 'private->number' being 6 (how can this
happen??); the next setsockopt() call for the same 'security' table
happens in about 1 minute with both the numbers being 6.
Can you please shed some light on the issue? Thanks!
BTW, iptables does have a retry mechanism for getsockopt():
2f93205b375e ("Retry ruleset dump when kernel returns EAGAIN.")
(https://git.netfilter.org/iptables/commit/libiptc?id=2f93205b375e=10=0=0)
But it looks like this is enough? e.g. here getsockopt() returns 0, but
setsockopt() returns -EAGAIN.
Thanks,
Dexuan
** Affects: linux-azure (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1928269
Title:
[Bug 1904632] Re: Ubuntu 18.04 Azure VM host kernel panic
Sure, will do. But AFAICT, there is no ETA yet. Even if the fix was made today, it would take quite some time (at least a few months?) to deploy the fix to the whole Azure fleet. :-( -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1904632 Title: Ubuntu 18.04 Azure VM host kernel panic To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1904632/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1904632] Re: Ubuntu 18.04 Azure VM host kernel panic
VM exits are pretty frequent and normal. "VM exits occur in response to certain instructions and events in VMX non-root operation" (see CHAPTER 27 VM EXITS of https://software.intel.com/content/www/us/en/develop/download/intel-64-and-ia-32-architectures-sdm-volume-3c-system-programming-guide-part-3.html. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1904632 Title: Ubuntu 18.04 Azure VM host kernel panic To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1904632/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1904632] Re: Ubuntu 18.04 Azure VM host kernel panic
VM Exit is a term in the Intel CPU's Virtualization support (VMX). It means the execution of the guest CPU is interrupted and the execution "jumps" to some function in the hypervisor; the hypervisor analyzes the reason of the VM Exit, and handles the VM exit properly, and then the execution "jumps" back to wherever the guest CPU was interrupted. Here the issue is: when the Level-2 guest CPU's VM Exit happens, somehow the hypervisor messes up the Level-1 guest's 32-bit related state (i.e. the SYSENTER instruction related state), so later when the 32-bit progarm starts to run, the Level-1 guest kernel crashes due to double-fault. The investigation is still ongoing. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1904632 Title: Ubuntu 18.04 Azure VM host kernel panic To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1904632/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1904632] Re: Ubuntu 18.04 Azure VM host kernel panic
Hyper-V team just identified a bug where the Hyper-V hypervisor can truncate the host SYSENTER_ESP/EIP to 16 bits on VMexit for some reason. A further investigation is ongoing. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1904632 Title: Ubuntu 18.04 Azure VM host kernel panic To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1904632/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1902531] Re: [linux-azure] IP forwarding issue in netvsc
Thanks, Marcelo! I tested all the 3 kernels and they worked as we expected. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1902531 Title: [linux-azure] IP forwarding issue in netvsc To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1902531/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1902531] Re: [linux-azure] IP forwarding issue in netvsc
This is the network config. Let me know if you need more info. ** Attachment added: "network-config.png" https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1902531/+attachment/5430820/+files/network-config.png -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1902531 Title: [linux-azure] IP forwarding issue in netvsc To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1902531/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1902531] Re: [linux-azure] IP forwarding issue in netvsc
To use Azure UDR, I referred to this page: https://campus.barracuda.com/product/cloudgenfirewall/doc/72516173/how- to-configure-azure-route-tables-udr-using-azure-portal-and-arm/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1902531 Title: [linux-azure] IP forwarding issue in netvsc To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1902531/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1902531] Re: [linux-azure] IP forwarding issue in netvsc
Here is how I reproduce the bug: Create 3 Ubuntu 16.04 VMs (VM-1, VM gateway, VM-2) on Azure in the same Resource Group. The kernel should be the linux-azure kernel 4.15.0-1098.109~16.04.1 (or newer). I use Gen1 VM but Gen2 should also has the same issue; I use the "East US 2" region, but the issue should reproduce in any region. Note: none of the VMs use Accelerated Networking, i.e. all of the 3 VMs use the software NIC "NetVSC". In my setup, VM-1 and VM-2 are "Standard D8s v3 (8 vcpus, 32 GiB memory)", and VM-gateway is "Standard DS3 v2 (4 vcpus, 14 GiB memory)". I happen to name the gateway VM "decui-dpdk", but here actually DPDK is not used at all (I do intend to use this setup for DPDK in future). The gateway VM has 3 NICs: The main NIC (10.0.0.4) is not used in ip-forwarding. NIC-1's IP is 192.168.80.5 NIC-2's IP is 192.168.81.5. The gateway VM receives packets from VM-1(192.168.80.4) and forwards the packets to VM-2 (192.168.81.4). No firewall rule is used. The client VM (VM-1, 192.168.80.4) has 1 NIC. It's running iperf2 client. The server VM (VM-2, 192.168.81.4) has 1 NIC. It's running iperf2 server: "nohup iperf -s &" The client VM is sending traffic, through the gateway VM (192.168.80.5, 192.168.81.5), to the server VM. Note: all the 3 subnets here are in the same VNET(Virtual Net) and 2 Azure UDR (User Defined Routing) rules must be used to force the traffic to go through the gateway VM. The IP-forwarding of the gateway VM's NIC-1 and NIC-2 must be enabled from Azure portal (the setting can only changed when the VM is "stopped"), and IP-forwarding must be enabled in the gateway VM (i.e. echo 1 > /proc/sys/net/ipv4/ip_forward). I'll attach some screenshots showing the network topology and the configuration. iperf2 uses 512 TCP connections and I limit the bandwidth used by iperf to <=70% of the per-VM bandwith limit (Note: if the VM uses >70% of the limit, even with the 2 patches, the ping latency between VM-1 and VM-2 can still easily go very high, e.g. >200ms -- we'll try to further investigate that). It looks the per-VM bandwithd limit of the gateway VM (DS3_v2) is 2.6Gbps, so 70% of it is 1.8Gbps. In the client VM, run something like: iperf -c 192.168.81.4 -b 3.5m -t 120 -P512 (-b means the per-TCP-connection limit; -P512 means 512 connections, so the total throughput should be around 3.5*512 = 1792 Mbps; "-t 120" means the test lasts for 2 minutes. we can abort the test any time by Ctrl+C.) In the "Server VM, run: nohup iperf -s & ping 192.168.80.4 (we can terminate the program by Ctrl+C), and observe the latency. In the gateway VM, run "nload" to check the current throughput (if the current device is not the NIC we want to check, press Right Arrow and Left Allow), and run "top" to check the CPU utilization (when there are 512 connections, the utilization should be still low, e.g. <25%). When the iperf2 test is running, the ping latency between VM-1 and VM-2 can easily exceed 100ms or even >300ms, but with the 2 patches applied, the latency typically should be <20ms. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1902531 Title: [linux-azure] IP forwarding issue in netvsc To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1902531/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1902531] Re: [linux-azure] IP forwarding issue in netvsc
I'll provide the instructions to reproduce the bug on Azure. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1902531 Title: [linux-azure] IP forwarding issue in netvsc To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1902531/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1902531] Re: [linux-azure] IP forwarding issue in netvsc
Since the 5.0 linux-azure kernel is not maintained anymore, IMO we don't have to fix this bug for it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1902531 Title: [linux-azure] IP forwarding issue in netvsc To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1902531/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894895] Re: [linux-azure][hibernation] ]VM hangs after hibernation/resume if the VM has SRIOV NIC and has been deallocated
The fix is in the mainline kernel now: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=19873eec7e13fda140a0ebc75d6664e57c00bfb1 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894895 Title: [linux-azure][hibernation] ]VM hangs after hibernation/resume if the VM has SRIOV NIC and has been deallocated To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1894895/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894893] Re: [linux-azure][hibernation] GPU device no longer working after resume from hibernation in NV6 VM size
The fix is in the PCI tree now: "PCI: hv: Fix hibernation in case interrupts are not re-create" ( https://git.kernel.org/pub/scm/linux/kernel/git/lpieralisi/pci.git/commit/?h=pci/hv=915cff7f38c5e4d47f187f8049245afc2cb3e503 ) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894893 Title: [linux-azure][hibernation] GPU device no longer working after resume from hibernation in NV6 VM size To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1894893/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894896] Re: [linux-azure][hibernation] Mellanox CX4 NIC's TX/RX packets stop increasing after hibernation/resume
We also need the second and the third patch: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=de214e52de1bba5392b5b7054924a08dbd57c2f6 https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=da26658c3d7005aa67a706dceff7b2807b59e123 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894896 Title: [linux-azure][hibernation] Mellanox CX4 NIC's TX/RX packets stop increasing after hibernation/resume To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1894896/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894896] [NEW] [linux-azure][hibernation] Mellanox CX4 NIC's TX/RX packets stop increasing after hibernation/resume
Public bug reported: Description of problem: In a VM with CX4 VF NIC on Azure, after hibernation/resume, the TX/RX packet counters stop increaseing. This issue doesn't exist in VM with a CX3 VF NIC. This happens to the latest stable release of the linux-azure 5.4.0-1023.23 kernel and the latest mainline linux kernel. How reproducible: 100% Steps to Reproduce: 1. Start a VM in Azure that supports Accelerated Networking, and enable hibernation properly (please refer to https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/comments/14 ). Please make sure the VF NIC is CX-4 since the issue doesn't happen to CX-3. 2. Do hibernation from serial console # systemctl hibernate 3. After the VM resumes back, check the MSI interrupt counters in /proc/interrupts for the CX-4 NIC, and also check “ifconfig” (e.g. “ifconfig enP2642s2”) for the RX/TX counters. These counters stop increasing while they should. BUG FIX: The fix is in the net.git tree now: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=19162fd4063a3211843b997a454b505edb81d5ce ** Affects: linux-azure (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894896 Title: [linux-azure][hibernation] Mellanox CX4 NIC's TX/RX packets stop increasing after hibernation/resume To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1894896/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894895] [NEW] [linux-azure][hibernation] ]VM hangs after hibernation/resume if the VM has SRIOV NIC and has been deallocated
Public bug reported: Description of problem: On Azure, if the VM is Stopped(deallocated) and later Started, the VF NIC's VMBus Instance GUID may change, and as a result hibernation/resume can hang forever. This happens to the latest stable release of the linux-azure 5.4.0-1023.23 kernel and the latest mainline linux kernel. How reproducible: 100% Steps to Reproduce: 1. Start a VM in Azure that supports Accelerated Networking, and enable hibernation properly (please refer to https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/comments/14 ) 2. Do hibernation from serial console # systemctl hibernate 4. After the VM state changes to "Stopped", click "Stop" button from Azure portal to change the VM state to Stopped(deallocated) 5. Wait for some time (e.g. 10 minutes? 1 hour?), and click the "Start" button to start the VM, and then check the boot-up process from the serial console. Actual results: Can not boot up. VM hangs after resume. Starting Resume from hibernation us…6c7-2c0c-491e-adcf-b625d69faf76... [ 19.822747] PM: resume from hibernation [ 19.836693] Freezing user space processes ... (elapsed 0.003 seconds) done. [ 19.846968] OOM killer disabled. [ 19.850236] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 20.542934] PM: Using 1 thread(s) for decompression [ 20.548250] PM: Loading and decompressing image data (559580 pages)... [ 22.844964] PM: Image loading progress: 0% [ 28.131327] PM: Image loading progress: 10% [ 32.346480] PM: Image loading progress: 20% [ 37.453971] PM: Image loading progress: 30% [ 40.834525] PM: Image loading progress: 40% [ 42.980629] PM: Image loading progress: 50% [ 44.342959] PM: Image loading progress: 60% [ 45.506197] PM: Image loading progress: 70% [ 46.800445] PM: Image loading progress: 80% [ 48.010185] PM: Image loading progress: 90% [ 49.045671] PM: Image loading done [ 49.050419] PM: Read 2238320 kbytes in 28.48 seconds (78.59 MB/s) [ 49.074198] printk: Suspending console(s) (use no_console_suspend to debug) (The VM hangs here forever) BUG FIX: A workaround patch is available and is being reviewed: https://lkml.org/lkml/2020/9/4/1270 ** Affects: linux-azure (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894895 Title: [linux-azure][hibernation] ]VM hangs after hibernation/resume if the VM has SRIOV NIC and has been deallocated To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1894895/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894893] [NEW] [linux-azure][hibernation] GPU device no longer working after resume from hibernation in NV6 VM size
Public bug reported: There are failed logs after resume from hibernation in NV6 (GPU passthrough size) VM in Azure: [ 1432.153730] hv_pci 47505500-0001--3130-444531334632: hv_irq_unmask() failed: 0x5 [ 1432.167910] hv_pci 47505500-0001--3130-444531334632: hv_irq_unmask() failed: 0x5 This happens to the latest stable release of the linux-azure 5.4.0-1023.23 kernel and the latest mainline linux kernel. How reproducible: 100% Steps to Reproduce: 1. Start a Standard_NV6 VM in Azure and enable hibernation properly (please refer to https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/comments/14 ) E.g. here I create a Generation-1 Ubuntu 20.04 Standard NV6_Promo (6 vcpus, 56 GiB memory) VM in East US 2. 2. Make sure the in-kernel open-source nouveau driver is loaded, or blacklist the nouveau driver and install the official Nvidia GPU driver (please follow https://docs.microsoft.com/en-us/azure/virtual- machines/linux/n-series-driver-setup : "Install GRID drivers on NV or NVv3-series VMs" -- the most important step to run the "./NVIDIA-Linux- x86_64-grid.run".) 3. Run hibernation from serial console # systemctl hibernate 4. After hibernation finishes, start VM and check dmesg # dmesg|grep fail Actual results: [ 1432.153730] hv_pci 47505500-0001--3130-444531334632: hv_irq_unmask() failed: 0x5 [ 1432.167910] hv_pci 47505500-0001--3130-444531334632: hv_irq_unmask() failed: 0x5 And /proc/interrupts shows that the GPU interrupts are no longer happening. Expected results: No failed logs, and the GPU interrupt should still happen after hibernation. BUG FIX: I made a fix here: https://lkml.org/lkml/2020/9/4/1268. Without the patch, we see the error "hv_pci 47505500-0001--3130-444531334632: hv_irq_unmask() failed: 0x5" during hibernation when the VM has the Nvidia GPU driver loaded, and after hibernation the GPU driver can no longer receive any MSI/MSI-X interrupts when we check /proc/interrupts. With the patch, we should no longer see the error, and the GPU driver should still receive interrupts after hibernation. ** Affects: linux-azure (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894893 Title: [linux-azure][hibernation] GPU device no longer working after resume from hibernation in NV6 VM size To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1894893/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891931] Re: [linux-azure] Panic when triggering hibernation
I can confirm now hibernation can work with 5.4.0-1023, despite a
harmless warning:
root@decui-tmp-2004:~# echo disk >/sys/power/state
[ 56.945758] PM: hibernation entry
[ 57.165520] Filesystems sync: 0.007 seconds
[ 57.169492] Freezing user space processes ... (elapsed 0.001 seconds) done.
[ 57.177529] OOM killer disabled.
[ 57.180702] PM: Marking nosave pages: [mem 0x-0x0fff]
[ 57.185925] PM: Marking nosave pages: [mem 0x0009f000-0x000f]
[ 57.191239] PM: Marking nosave pages: [mem 0x3fff-0x]
[ 57.197810] PM: Basic memory bitmaps created
[ 57.201563] PM: Preallocating image memory... done (allocated 210160 pages)
[ 57.623616] PM: Allocated 840640 kbytes in 0.41 seconds (2050.34 MB/s)
[ 57.629195] Freezing remaining freezable tasks ... (elapsed 0.000 seconds)
done.
[ 57.637795] serial 00:04: disabled
[ 58.847939] Disabling non-boot CPUs ...
[ 58.852140] smpboot: CPU 1 is now offline
[ 58.857921] smpboot: CPU 2 is now offline
[ 58.863623] smpboot: CPU 3 is now offline
[ 58.869363] unchecked MSR access error: WRMSR to 0x4106 (tried to write
0x412d4f49 000100ee) at rIP: 0x9ee1d9b8 (hv_cpu_die+0xe8/0x110)
[ 58.870052] Call Trace:
[ 58.870052] hv_suspend+0x5a/0x87
[ 58.870052] syscore_suspend+0x59/0x1a0
[ 58.870052] hibernation_snapshot+0x1bc/0x460
[ 58.870052] hibernate.cold+0x6d/0x1f6
[ 58.870052] state_store+0xde/0xe0
[ 58.870052] kobj_attr_store+0x12/0x20
[ 58.870052] sysfs_kf_write+0x3e/0x50
[ 58.870052] kernfs_fop_write+0xda/0x1b0
[ 58.870052] __vfs_write+0x1b/0x40
[ 58.870052] vfs_write+0xb9/0x1a0
[ 58.870052] ksys_write+0x67/0xe0
[ 58.870052] __x64_sys_write+0x1a/0x20
[ 58.870052] do_syscall_64+0x5e/0x200
[ 58.870052] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 58.870052] RIP: 0033:0x7f2f9dfcb057
[ 58.870052] Code: 64 89 02 48 c7 c0 ff ff ff ff eb bb 0f 1f 80 00 00 00 00
f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00
f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[ 58.870052] RSP: 002b:7ffe96046608 EFLAGS: 0246 ORIG_RAX:
0001
[ 58.870052] RAX: ffda RBX: 0005 RCX: 7f2f9dfcb057
[ 58.870052] RDX: 0005 RSI: 55ca5250c450 RDI: 0001
[ 58.870052] RBP: 55ca5250c450 R08: 000a R09: 0004
[ 58.870052] R10: 55ca50a2d017 R11: 0246 R12: 0005
[ 58.870052] R13: 7f2f9e0a66a0 R14: 7f2f9e0a74a0 R15: 7f2f9e0a68a0
[ 58.870052] PM: Creating hibernation image:
[ 58.870052] PM: Need to copy 201788 pages
[ 58.870052] PM: Normal pages needed: 201788 + 1024, available pages: 3992087
[ 58.870052] PM: Hibernation image created (201788 pages copied)
[ 58.870052] Enabling non-boot CPUs ...
[ 58.870052] x86: Booting SMP configuration:
[ 58.871862] smpboot: Booting Node 0 Processor 1 APIC 0x1
[ 58.875719] CPU1 is up
[ 58.877194] smpboot: Booting Node 0 Processor 2 APIC 0x2
[ 58.881047] CPU2 is up
[ 58.882499] smpboot: Booting Node 0 Processor 3 APIC 0x3
[ 58.886033] CPU3 is up
[ 58.891099] hv_utils: KVP IC version 4.0
[ 58.893181] hv_utils: Shutdown IC version 3.2
[ 58.896580] hv_balloon: Using Dynamic Memory protocol version 2.0
[ 60.186366] hv_utils: Heartbeat IC version 3.0
[ 61.952674] hv_utils: TimeSync IC version 4.0
[ 68.108243] hv_balloon: Max. dynamic memory size: 16384 MB
[ 70.552511] serial 00:03: activated
[ 70.620778] serial 00:04: activated
[ 70.692760] PM: Using 3 thread(s) for compression
[ 70.716148] ata1.01: host indicates ignore ATA devices, ignored
[ 70.760736] PM: Compressing and saving image data (202183 pages)...
[ 70.760749] PM: Image saving progress: 0%
[ 70.831492] ata1.00: host indicates ignore ATA devices, ignored
[ 74.568857] PM: Image saving progress: 10%
[ 89.707652] PM: Image saving progress: 20%
[ 109.659651] PM: Image saving progress: 30%
[ 125.565315] PM: Image saving progress: 40%
[ 140.112605] PM: Image saving progress: 50%
[ 146.074334] PM: Image saving progress: 60%
[ 152.507964] PM: Image saving progress: 70%
[ 161.068827] PM: Image saving progress: 80%
[ 170.115167] PM: Image saving progress: 90%
[ 177.616417] PM: Image saving progress: 100%
[ 178.566922] PM: Image saving done
[ 178.623924] PM: Wrote 808732 kbytes in 107.80 seconds (7.50 MB/s)
[ 178.686742] PM: S|
[ 178.791430] kvm: exiting hardware virtualization
[ 178.851852] sd 0:0:0:0: [sdb] Synchronizing SCSI cache
[ 178.913444] ACPI: Preparing to enter system sleep state S5
[ 178.975244] reboot: Power down
[ 179.043250] acpi_power_off called
This warning can be fixed by this upstream fix:
38dce4195f0d ("x86/hyperv: Properly suspend/resume reenlightenment
notifications")
How to reproduce the warning: before following
https://bugs.launchpad.net/ubuntu/+source/linux-
azure/+bug/1880032/comments/14 to test hibernation, make sure that
"lsmod"
[Bug 1888715] Re: UDP data corruption caused by buggy udp_recvmsg() -> skb_copy_and_csum_datagram_msg()
FYI: the fix is in the upstream linux-4.4.y branch now: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=v4.4.233=c514bb4147e2c667cf82f9aa7689cf442078c13f -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1888715 Title: UDP data corruption caused by buggy udp_recvmsg() -> skb_copy_and_csum_datagram_msg() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1888715/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880032] Re: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
Detailed steps to repro the issueo on Azure: 1. Create a VM with the image "Ubuntu Server 20.04 LTS - Gen1". Any VM size should be fine. Here I use "Standard E4-2ds_v4 (2 vcpus, 32 GiB memory)". 2. Add an extra disk of 64GB to the VM via Azure portal. 3. Login the VM via ssh and check the kernel version: here I get 5.4.0-1022-azure. 4. In the VM, the 64GB disk can be sdc. Let's create a swap partition in it, i.e. sdc1. 5. mkswap /dev/sdc1 root@decui-tmp-2004:~# mkswap /dev/sdc1 Setting up swapspace version 1, size = 64 GiB (68718424064 bytes) no label, UUID=544831e4-72ab-4d2c-81aa-6dac3a8e20ad 6. Add the swap partition info into /etc/fstab: UUID=544831e4-72ab-4d2c-81aa-6dac3a8e20ad none swap sw 0 0 7. Use "swapon -a; swapon -s" to confirm that the swap partition works. 8. Add the kernel parameter resume= into /etc/default/grub.d/50-cloudimg-settings.cfg: GRUB_CMDLINE_LINUX_DEFAULT="console=tty1 console=ttyS0 earlyprintk=ttyS0 resume=UUID=544831e4-72ab-4d2c-81aa-6dac3a8e20ad ignore_loglevel no_console_suspend" Note: here I also add "ignore_loglevel no_console_suspend", which are *required* to see the error messages during hibernation. 9. Comment out the only line in /etc/default/grub.d/40-force-partuuid.cfg: GRUB_FORCE_PARTUUID=bf00dea3-136e-49cb-a640-0df7ce49d6db Note: this step is required, otherwise the generated grub.cfg doesn't contain the "initrd ..." line , which is required for resuming to work. 10. Run "update-grub2; reboot". Note: this 'reboot' might be a must, because we'll need to re-generate the initramfs when the running kernel has the resume= parameter. 11. Login the VM again and run "update-initramfs -u". 12. Run "echo disk > /sys/power/state". Note: we'd better run this command from Azure serial console (we need to set a password for root and use that to login via the serial console) so we can easily watch what will be happening. root@decui-tmp-2004:~# echo disk > /sys/power/state [ 67.838749] PM: hibernation entry [ 68.266627] Filesystems sync: 0.041 seconds [ 68.271740] Freezing user space processes ... (elapsed 0.001 seconds) done. [ 68.281528] OOM killer disabled. [ 68.286475] PM: Marking nosave pages: [mem 0x-0x0fff] [ 68.293459] PM: Marking nosave pages: [mem 0x0009f000-0x000f] [ 68.300306] PM: Marking nosave pages: [mem 0x3fff-0x] [ 68.308250] PM: Basic memory bitmaps created [ 68.313082] PM: Preallocating image memory... done (allocated 298659 pages) [ 69.303864] PM: Allocated 1194636 kbytes in 0.98 seconds (1219.01 MB/s) [ 69.311605] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 69.322486] serial 00:04: disabled [ 69.345193] [ cut here ] [ 69.345199] WARNING: CPU: 1 PID: 1495 at kernel/workqueue.c:3040 __flush_work+0x1b5/0x1d0 ... [ 70.047238] CPU1 is up [ 70.054474] hv_utils: KVP IC version 4.0 [ 70.056763] hv_utils: Shutdown IC version 3.2 [ 70.061009] hv_balloon: Using Dynamic Memory protocol version 2.0 It looks the kernel hangs here forever. Normally the VM is expected to save the state to disk and power off and later when we start the VM from the portal, the VM is expected to resume back from the 'echo' command on the serial console. If I build a kernel with the same source code but revert 0a14dbaa0736, the above suspending and resuming work fine. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880032 Title: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880032] Re: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
Hi Marcelo, yes, please revert
0a14dbaa0736 ("video: hyperv_fb: Fix hibernation for the deferred IO feature").
No other change is needed.
In the future, when a4ddb11d297e is included, 0a14dbaa0736 should also
be included.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1880032
Title:
[linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880032] Re: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
To reproduce the issue, I created a Ubuntu 20.04 VM on Azure (the kernel version was "5.4.0-1022-azure #22-Ubuntu"), and I ran "echo disk > /sys/power/state" in the VM and then checked the Azure serial console of the VM and found the warning in commen #8 and suspending couldn't finish normally (it looks the VM got a fatal page fault error later). I suppose the issue can also repro on a local Hyper-V host. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880032 Title: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880032] Re: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
Unluckily this commit breaks hibernation:
0a14dbaa0736 ("video: hyperv_fb: Fix hibernation for the deferred IO feature"):
https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-azure/+git/focal/commit/?h=Ubuntu-azure-5.4.0-1022.22=0a14dbaa0736a6021c02e74d42cf3a7ca5438da6
The kernel here doesn't include
a4ddb11d297e ("video: hyperv: hyperv_fb: Support deferred IO for Hyper-V frame
buffer driver", so it should not include
0a14dbaa0736 ("video: hyperv_fb: Fix hibernation for the deferred IO feature").
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1880032
Title:
[linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880032] Re: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
Unluckily this commit breaks hibernation:
0a14dbaa0736 ("video: hyperv_fb: Fix hibernation for the deferred IO feature"):
https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-azure/+git/focal/commit/?h=Ubuntu-azure-5.4.0-1022.22=0a14dbaa0736a6021c02e74d42cf3a7ca5438da6
We should include the patch only if the kernel also includes
a4ddb11d297e ("video: hyperv: hyperv_fb: Support deferred IO for Hyper-V frame
buffer driver"
Now I'm seeing a hang/panic issue when hibernating the VM ("5.4.0-1022-azure
#22-Ubuntu"):
[ 67.736061] [ cut here ]
[ 67.736068] WARNING: CPU: 5 PID: 1358 at kernel/workqueue.c:3040
__flush_work+0x1b5/0x1d0
[ 67.736068] Modules linked in: xt_owner iptable_security xt_conntrack
nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c bpfilter nls_iso8859_1
dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua sb_edac crct10dif_pclmul
crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd glue_helper
joydev hid_generic hyperv_fb cfbfillrect hid_hyperv intel_rapl_perf serio_raw
hyperv_keyboard pata_acpi hv_netvsc hv_balloon hid cfbimgblt pci_hyperv
cfbcopyarea hv_utils pci_hyperv_intf sch_fq_codel drm
drm_panel_orientation_quirks i2c_core ip_tables x_tables autofs4
[ 67.736088] CPU: 5 PID: 1358 Comm: bash Not tainted 5.4.0-1022-azure
#22-Ubuntu
[ 67.736089] Hardware name: Microsoft Corporation Virtual Machine/Virtual
Machine, BIOS 090007 06/02/2017
[ 67.736091] RIP: 0010:__flush_work+0x1b5/0x1d0
[ 67.736092] Code: f0 eb e3 4d 8b 7c 24 20 e9 f3 fe ff ff 8b 0b 48 8b 53 08
83 e1 08 48 0f ba 2b 03 80 c9 f0 e9 4f ff ff ff 0f 0b e9 68 ff ff ff <0f> 0b 45
31 f6 e9 5e ff ff ff e8 ec e0 fd ff 66 66 2e 0f 1f 84 00
[ 67.736095] RSP: 0018:a7ce8a8ffb78 EFLAGS: 00010246
[ 67.736096] RAX: RBX: 8be3621f02a0 RCX:
[ 67.736096] RDX: 0001 RSI: 0001 RDI: 8be3621f02a0
[ 67.736097] RBP: a7ce8a8ffbf0 R08: R09: ff010101
[ 67.736098] R10: 8be363f7a320 R11: 0001 R12: 8be3621f02a0
[ 67.736098] R13: 0001 R14: 0001 R15: bc390fd1
[ 67.736099] FS: 7f6df35fe740() GS:8be375d4()
knlGS:
[ 67.736100] CS: 0010 DS: ES: CR0: 80050033
[ 67.736100] CR2: 561eef2c1b50 CR3: 000e40a14004 CR4: 001706e0
[ 67.736102] Call Trace:
[ 67.736108] __cancel_work_timer+0x107/0x180
[ 67.736119] cancel_delayed_work_sync+0x13/0x20
[ 67.736121] hvfb_suspend+0x48/0x80 [hyperv_fb]
[ 67.736122] vmbus_suspend+0x2a/0x40
[ 67.736125] dpm_run_callback+0x5b/0x150
[ 67.736127] __device_suspend_noirq+0x9e/0x2f0
[ 67.736128] dpm_suspend_noirq+0x101/0x2d0
[ 67.736130] dpm_suspend_end+0x53/0x80
[ 67.736132] hibernation_snapshot+0xd8/0x460
[ 67.736133] hibernate.cold+0x6d/0x1f6
[ 67.736135] state_store+0xde/0xe0
[ 67.736138] kobj_attr_store+0x12/0x20
[ 67.736141] sysfs_kf_write+0x3e/0x50
[ 67.736142] kernfs_fop_write+0xda/0x1b0
[ 67.736145] __vfs_write+0x1b/0x40
[ 67.736147] vfs_write+0xb9/0x1a0
[ 67.736149] ksys_write+0x67/0xe0
[ 67.736150] __x64_sys_write+0x1a/0x20
[ 67.736152] do_syscall_64+0x5e/0x200
[ 67.736156] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 67.736157] RIP: 0033:0x7f6df3712057
After I revert 0a14dbaa0736, hibernation works.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1880032
Title:
[linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1888715] Re: UDP data corruption caused by buggy udp_recvmsg() -> skb_copy_and_csum_datagram_msg()
https://lore.kernel.org/netdev/20200728015505.37830-1-de...@microsoft.com/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1888715 Title: UDP data corruption caused by buggy udp_recvmsg() -> skb_copy_and_csum_datagram_msg() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1888715/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1888715] Re: UDP data corruption caused by buggy udp_recvmsg() -> skb_copy_and_csum_datagram_msg()
rcu_access_pointer(sk->sk_filter) is basically the same as sk->sk_filter. If sk->sk_filter is true, the change makes no difference. If sk->sk_filter is false, the change also drops a UDP packet with incorrect UDP checksum by "goto csum_error;". Without the change, the packet is dropped in udp_recvmsg(); with the change, the packet is dropped earlier. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1888715 Title: UDP data corruption caused by buggy udp_recvmsg() -> skb_copy_and_csum_datagram_msg() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1888715/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880032] Re: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
FYI: the patch "net/mlx5: Fix crash upon suspend/resume" is in v5.7 now (i.e. today's latest mainline): https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.7=8fc3e29be9248048f449793502c15af329f35c6e -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880032 Title: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1880032] Re: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels
There is another important bug fix for hibernation: net/mlx5: Fix crash upon suspend/resume (https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=8fc3e29be9248048f449793502c15af329f35c6e). So far the fix is only present in the net.git tree, but I expect it will be in the mainline tree’s v5.8-rc1 (or even v5.7, if we’re lucky). Please consider picking it up. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1880032 Title: [linux-azure] Enable Hibernation on The 18.04 and 20.04 5.4 Kernels To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1880032/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
So it looks this is considered as a feature rather than a bug for the Ubuntu 20.04 VM image in Azure Marketplacet. To whoever uses such an image on Azure: if you're installing a new kernel that doesn't have the necessary drivers built-in (CONFIG_HYPERV=y, CONFIG_HYPERV_STORAGE=y), you're supposed to comment out the GRUB_FORCE_PARTUUID line in /etc/default/grub.d/40-force-partuuid.cfg and run 'sudo update-grub'. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
BTW, the symptom described in the Bug Description also exists in the Ubuntu 20.04 image in Azure Marketplace. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
> If someone is using a kernel other than the one we provide for the cloud, or in the case of a bug, the system will still boot (slower) after a panic and a reboot to try again with the initrd. Hi Steve, I guess you assume the pattern is "panic/success/success/success/...", but actually the pattern is “panic/success/panic/success/panic/success/...” -- this is pretty confusing. Please refer to the the Bug Description for details. Ideally grub should be configured to not add the 'initrd' line only for the cloud kernels. Is there a way for grub to tell if the kernel is a cloud kernel or not? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
Anyone knows who maintains the grub package shipped in the cloud-images? Should we report a bug at https://bugs.launchpad.net/ubuntu/+source/grub2 ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
I think commen #6 is correct: it looks the 2018 patch introduced the issue for us, but the patch is originally for "initrd-less boot capabilities." and here we do need the initramfs file. I guess the patch "ubuntu-add-initrd-less-boot-fallback.patch" is not included into the grub shipped in the 20.04 .iso file ubuntu-20.04-live- server-amd64.iso, but somehow it's included into the grub shipped in the cloud-image? If so, I guess we can fix this bug by removing the patch for cloud-image? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
Today I just repeated the test "Create a Gen-1 Ubuntu 19.10 VM on Azure,
and upgrade it to Ubuntu 20.04 by “do-release-upgrade –d" and I
reproduced this bug again, and the grub version is also 2.04-1ubuntu26!
So I suspect grub itself should be good, but some grub config file (i.e.
/etc/grub.d/10_linux?) causes the bug?
I checked my /etc/grub.d/10_linux: after I added line 263, "grub-mkconfig" can
generate the needed initrd line correctly:
257 fi
258
259 sed "s/^/$submenu_indentation/" << EOF
260 initrd${rel_dirname}/${initrd}
261 else
262 linux ${rel_dirname}/${basename}
root=${linux_root_device_thisversion} ro ${args} panic=-1
263 initrd${rel_dirname}/${initrd}
264 fi
265 initrdfail
266 EOF
My /etc/grub.d/10_linux is from the grub2-common package
(2.04-1ubuntu26). It looks this file in my VM that's upgraded from 19.10
to 20.04 is different from the version of the file in a VM that's
created from https://releases.ubuntu.com/20.04/ubuntu-20.04-live-
server-amd64.iso
That's why I suspected it is specific to the cloud-image version of
Ubuntu 20.04. I don't know how exactly “do-release-upgrade -d" works and
where the upgrade procedure pulls the grub2 that lacks the initrd line
in the /etc/grub.d/10_linux.
In summary,
1.
https://cloud-images.ubuntu.com/focal/20200430.1/focal-server-cloudimg-amd64.img
and
https://cloud-images.ubuntu.com/focal/20200430.1/focal-server-cloudimg-amd64-azure.vhd.zip
have the bug.
2. https://releases.ubuntu.com/20.04/ubuntu-20.04-live-server-amd64.iso does
not have the bug.
3. A quick fix is add the needed line 263 (see above), but I think we need to
understand how the bug is introduced.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1870189
Title:
initramfs does not get loaded
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
Sorry, this statement is wrong: == Today I also created a VM on my host from https://cloud-images.ubuntu.com/focal/20200430.1/focal-server-cloudimg-amd64-azure.vhd.zip and can not see the bug either, and the grub version is also 2.04-1ubuntu26. == Actually I do see the bug as well with the vhd.zip file. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
Today I installed a Generation-1 Ubuntu 20.04 VM on my local Hyper-V host from the ISO file: https://releases.ubuntu.com/20.04/ubuntu-20.04 -live-server-amd64.iso (released on 4/23/2020) and I don't see this bug and the grub version is 2.04-1ubuntu26. Today I also created a VM on my host from https://cloud- images.ubuntu.com/focal/20200430.1/focal-server-cloudimg- amd64-azure.vhd.zip and can not see the bug either, and the grub version is also 2.04-1ubuntu26. When the bug was originally reported on Apr 1 against my Azure VM (a Ubuntu 19.10 VM on Azure was created and upgraded to Ubuntu 20.04 by “do-release-upgrade –d"), the grub version was 2.04-1ubuntu22. So it looks the issue has been fixed in the 26 version? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
When the bug was originally reported on Apr 1, "We installed a Ubuntu
20.04 VM from the .iso file from http://cdimage.ubuntu.com/daily-
live/pending/ and don’t see the strange grub issue". It looks the grub
version used in the .iso file (on Apr 1) does not have the bug.
I don't think the patch in the link mentioned in comment #6 causes the
bug, because that patch was made 2 years ago and we started to see this
bug just recently. Of course I can be wrong, since I don't really have a
lot of grub knowledge. :-)
I'm not sure if the commit 6a814c759e10 ("Import patches-unapplied
version 2.04-1ubuntu1 to ubuntu/eoan-proposed", made on Jul 16 11:31:29
201) causes this bug, either, since it's made almost 10 months ago.
Again, I can be wrong. :-) BTW, this commit is huge -- more than 12K
lines.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1870189
Title:
initramfs does not get loaded
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Thanks for the reminder! I just realized Ubuntu 20.02 was already released on 4/23. We should try it. For the CPU firmware (CPU microcode?) update issue: sorry, it's completely out of my scope -- I only work on Linux. Hopefully that issue will be resolved in the near future. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Sorry, I made a typo above: systemd.dsystemd.default_standard_output=kmsg ==> systemd.default_standard_output=kmsg. BTW, it looks systemd.show_status=true makes no difference for me. I don't see any status info during the boot-up time -- not sure if I did something wrong. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
I don't have much knowledge bout systemd, either :-) I just did a "man systemd" and found the options of systemd. "man systemd" says that we can use pass these kernel parameters to systemd: systemd.service_watchdogs=true systemd.show_status=true systemd.log_level=debug systemd.dsystemd.default_standard_output=kmsg systemd.default_standard_error=kmsg I tried these by adding them into /boot/grub/grub.cfg manually, at the end of the line "linux /boot/vmlinuz-5.3.0-23-generic ...". I also replaced "quiet splash $vt_handoff" with "ignore_loglevel". So I can get more messages from systemd, but not so much as I expected. Not sure if this would be helpful to troubleshoot the long delay issue for you, and I'm not even sure if I enabled the systemd loggong completely correctly -- again, I'm not really familiar with systemd. :-) To stop/disble a systemd "service", I think we can use something like this (taking the setvtrgb.service as an example): systemctl stop setvtrgb.service systemctl disable setvtrgb.service systemctl status setvtrgb.service -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Since Alt-SysRq-w gives nothing, I'm sure the long delay is not a
kernel/driver issue but a user space issue. It looks due to some reason
I just can not reproduce the long delay. :-(
In the Hyper-V Virtual Machine Connection window's "View" menu, there is
an item "Enhanced Session". In my Ubuntu 19.10 VM created by "Quick
Create...", the xrdp daemon/service is configured to automatically run
during the boot-up procedure; I think as soon as the xrdp daemon starts
to run, the "Enhanced Session" item becomes clickable/usable, and I can
click it to toggle between "Enhanced Session" mode (i.e. xrdp mode) and
the native Xorg GUI mode; when I'm in the xrdp mode, I click VM
Connection window's Action | Shut Down, then Start, and the VM will boot
up to the xrdp login screen in about 14 seconds; when I'm in the Xorg
mode, I click Shutdown then Start, the VM will boot up to the Xorg GUI
desktop in about 30 seconds. If I shut down the VM, close the VM
Connection window, and start the VM and open VM Connection window, I'll
be prompted by a small pop-up window to choose a resolution when (I
think) the xrdp daemon starts to run: 1) if I click the close icon of
the small window, I'll be in the Xorg GUI mode; if I accept the default
resolution (or change to a different resolution) and click "connect" in
the small window, I'll be in the xrdp mode. So all these work pretty
good for me.
Note: after I just created the 19.10 VM by "Quick Create..." and set up
the host name and user name/password stuff, I rebooted the VM and when
the VM booted up, I found the "Enhanced Session" was not
clickable/usable -- this looks like a bug -- while I still don't know
the root cause, it looks this can be resolved by manually adding the
line "initrd/boot/initrd.img-5.3.0-23-generic #This line is
added by Dexuan manually" into "/boot/grub/grub.cfg":
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os
$menuentry_id_option 'gnulinux-simple-55829715-0091-4b86-b060-1cb88f342faf' {
...
if [ "${initrdfail}" = 1 ]; then
linux /boot/vmlinuz-5.3.0-23-generic
root=PARTUUID=43e99d31-1277-402c-a13b-6cc8fb93169b ro quiet splash $vt_handoff
initrd/boot/initrd.img-5.3.0-23-generic
else
linux /boot/vmlinuz-5.3.0-23-generic
root=PARTUUID=43e99d31-1277-402c-a13b-6cc8fb93169b ro quiet splash $vt_handoff
panic=-1
initrd/boot/initrd.img-5.3.0-23-generic #This line is added
by Dexuan manually!!!
fi
initrdfail
}
With the addition of the line, it looks "GRUB_TIMEOUT=0" in "/etc/default/grub"
is always really applied every time I reboot the VM.
Without the line, it looks sometimes the grub timeout is 30 second and
sometimes it's 0 second.
BTW, I reported a bug for the missing initrd line a few weeks ago for a
different issue: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189.
I suggest you also manually add the line, then I guess you should be
able to reliably toggle between xrdp mode and Xorg mode.
Note: "/boot/grub/grub.cfg" is overwritten when update-grub is run by us
or some automatic-update daemon, so we may want to check if the line is
still there when we see something unexpected (i.e. unable to use xrdp
mode, or see a grub timeout of 30s). I hope Bug 1870189 will be fixed by
somebody ASAP...
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1848534
Title:
[Microsoft Hyper-V guest] System shows graphic artifacts for a moment,
then text cursor for about minute and then starts
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
I also tried xrdp mode and the VM booted up to the xrdp login window in 14 seconds, which is faster than the "native Xorg GUI mode" (which needs 30s) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Sorry, I did miss this part of your previous reply: root@stock19:~# systemctl list-jobs JOB UNIT TYPE STATE 48 setvtrgb.service start waiting 137 system-getty.slice start waiting 1 graphical.target start waiting 102 systemd-update-utmp-runlevel.service start waiting 83 plymouth-quit-wait.service start running 2 multi-user.target start waiting I'm wondering if you can disable setvtrgb.service, system-getty.slice, systemd-update-utmp-runlevel.service, and plymouth-quit-wait.service, and see if the long delay will disappear. I guess these 4 services don't look critical to the GUI desktop. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1870189] Re: initramfs does not get loaded
I agree with David. IMO this bug should be fixed ASAP. Thanks! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1870189 Title: initramfs does not get loaded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1870189/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
It looks #48 shows some service is causing the long delay -- can you try 'systemctl list-jobs' to see active jobs, as the "Hint" says? :-) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
I created a Ubuntu 19.10 VM via "Quick Create..." and still can not reproduce the long delay of > 1 minute: the VM can boot up to the Xorg GUI desktop in 26 seconds. My Windows 10 has the same version info: Version 1909 (OS Build 18363.778). At the grub screen, can you press 'e' and, manually edit the kernel parameter: please remove the "quiet splash $vt_handoff" and add "ignore_loglevel sysrq_always_enabled". You may want to enable the serial console logging by adding the kernel parameter "console=ttyS0", and attach putty (Run as Administrator) to the named pipe \\.\pipe\debug_slow_vm, assuming you configure the VM serial console by "Set-VMComPort -VMName your_vm_name -number 1 -path \\.\pipe\debug_slow_vm"). This way, you should get more messages on the VM serial console when the VM boots up. When you see the long delay, you can press SysRQ+w (i.e. the Right Alt + SysRq + w) to show the blocked processes, if any. This may provide more info about the long delay. BTW, here I assume your have a keyboard that has the SysRq key. :-) It looks systemd can be configured to use "--log-level=debug --default- standard-output=kmsg --default-standard-error=kmsg", which may provide more info as well, if we check 'dmesg' and/or the VM serial console. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
@msgallery: BTW, you mentioned 'The "restart" button is not functional' -- actually it is not functional only when we try to click the button by mouse -- if we press Tab to focus on the button and then press Enter, the VM should reboot. :-) I'll try to mention this to Hyper-V team, but I'm not sure when they will fix this minor issue, since the issue should be of low priority. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
BTW, my Linux kernel version is 5.3.0-46-generic #38-Ubuntu (17:37:05, 3/27/2020). The "graphic artifact" is somehow caused by the "$vt_hanoff" kernel parameter (check "cat /proc/cmdline"). If I manually remove the "$vt_hanoff" at the grub screen, I won't see the "graphic artifact" -- Ubuntu guys should take a look and fix the issue, as I'm not familiar with "vt_handoff". @msgallery: I never see the "1:40" (1 minutes and 40 seconds) delay reported in comment #40. Maybe you can use "systemd-analyze critical- chain" (mentioned in Comment #25) to figure out why the delay happened. To recap, my experience with the fresh Desktop installation of Ubuntu 19.10 (Gen-2 VM) on Hyper-V is good, except for the minor "graphic artifact" issue. I don't see any long delay. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
This is the screenshot of the graphic artifact mentioned in the previous comment. ** Attachment added: "graphic_artifact.png" https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+attachment/5352858/+files/graphic_artifact.png -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Today I installed a Generation-2 VM (4 virtual CPUS, 4 GB memory) from the this .iso file: http://releases.ubuntu.com/19.10/ubuntu-19.10-desktop-amd64.iso. My host is Win10: Version 1909 (OS Build 18383.720) -- I got the info by running the built-in "winver.exe" program. The CPU type is Intel Core I7-7600 (2.80G Hz). There are 2 cores and the cores support SMT, so there are 4 logical processors in total. I can see the "graphic artifact" (I will upload a screenshot soon), but it looks overall the boot-up is fast (it takes 30 seconds) and it looks the VM works fine for me. When the VM boots up: 1. First, the screen with the purple background (I think it's from grub) remains 4 seconds. 2. The screen background becomes black, and there is a "Hyper-V" logo in the center of the screen. This remains about 1 second. 3. The screen with the "graphic artifact" appears, and remains about 4 seconds. 4. The screen background becomes purple and the "Ubuntu" logo with 5 dots appears. This screen remains 8 seconds. 5. The screen becomes completely black. This screen remains 9 seconds. 6. The screen becomes kind of purple again, and in about 2 seconds the GUI desktop appears (I set Ubuntu to automatically login in to the desktop). So the overall time spent on the 6 steps are 30 seconds. IMO this looks normal. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/gdm3/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1867220] Re: Assignment of VDEV Somtimes Fails using Intel QAT
Hi Marcelo, I'm not sure which v5.3 kernel you mean -- the v5.3 in Ubuntu 19.10, v5.4 in Ubuntu 20.04 or the upstream stable tree's v5.3 and newer? :-) Here we need to make sure the 3 patches in the Bug Description are included, and also make sure the line "if (list_empty(>children)) hbus->sysdata.domain = desc->ser" in new_pcichild_device() should be completely removed. BTW, this line never made it into the upstream kernel and it only appears in some Ubuntu versions (if not all). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1867220 Title: Assignment of VDEV Somtimes Fails using Intel QAT To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1867220/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1869996] [NEW] Hibernation can NOT work due to incorrect RESUME=
Public bug reported:
In Ubuntu 18.04 I have the below 4 lines, but in Ubuntu 20.04 (dev
build) these lines are missing:
--- /usr/share/initramfs-tools/init2020-04-01 01:15:20.533208700 +
+++ /usr/share/initramfs-tools/init 2020-04-01 00:59:43.931655200 +
@@ -163,6 +163,10 @@
;;
resume=*)
RESUME="${x#resume=}"
+ case $RESUME in
+ UUID=*)
+RESUME="/dev/disk/by-uuid/${RESUME#UUID=}"
+ esac
;;
resume_offset=*)
resume_offset="${x#resume_offset=}"
As a result, hibernation can not work: the saved state in the swap
partition is lost and no resume happens.
The issue is: when I use the "resume=UUID=533b2cd9-31ac-449b-82ff-
014f09ab0a9c" kernel parameter for hibernation, due to the missing
lines, the variable 'resume' in a later place in "/usr/share/initramfs-
tools/init" is set to "UUID=533b2cd9-31ac-449b-82ff-014f09ab0a9c" rather
than "/dev/disk/by-uuid/533b2cd9-31ac-449b-82ff-014f09ab0a9c". Next, in
/usr/share/initramfs-tools/scripts/local-premount/resume:
DEV=$(readlink "$resume")
DEV=/sys/class/block/${DEV##*/}/dev
if [ -r "$DEV" ]; then
read -r MAJMIN < "$DEV"
fi
if [ -z "$MAJMIN" ]; then
exit 1
fi
Here the 'readlink' will fail, so $DEV is not pointing to a valid device, and
then $MAJMIN is empty and
we "exit 1", so no resume can happen!
Not sure why the 4 lines are removed in Ubuntu 20.04...
PS, this is my version info:
root@localhost:~# dpkg-query -s initramfs-tools-core
Package: initramfs-tools-core
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 271
Maintainer: Ubuntu Developers
Architecture: all
Multi-Arch: foreign
Source: initramfs-tools
Version: 0.136ubuntu1
Replaces: initramfs-tools (<< 0.121~)
Depends: busybox-initramfs (>= 1:1.30.1-4ubuntu5~), initramfs-tools-bin (=
0.136ubuntu1), klibc-utils (>= 2.0.4-8~), cpio (>= 2.12), lz4, kmod, udev,
coreutils (>= 8.24), logsave | e2fsprogs (<< 1.45.3-1~)
Suggests: bash-completion
Breaks: busybox-initramfs (<< 1:1.30.1-4ubuntu5~), initramfs-tools (<< 0.121~)
Conffiles:
/etc/initramfs-tools/initramfs.conf 4ec999d424d01b9ca685e65ba0f22a13
Description: generic modular initramfs generator (core tools)
This package contains the mkinitramfs program that can be used to
create a bootable initramfs for a Linux kernel. The initramfs should
be loaded along with the kernel and is then responsible for mounting
the root filesystem and starting the main init system.
Original-Maintainer: Debian kernel team
** Affects: initramfs-tools (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1869996
Title:
Hibernation can NOT work due to incorrect RESUME=
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/1869996/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1867220] Re: Assignment of VDEV Somtimes Fails using Intel QAT
BTW, the bug also applies to hwe-4.15.0-91.92_16.04.1 and Ubuntu- hwe-5.0.0-37.40_18.04. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1867220 Title: Assignment of VDEV Somtimes Fails using Intel QAT To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1867220/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1867220] Re: Assignment of VDEV Somtimes Fails using Intel QAT
The bug applies to both linux-azure-5.0.0-1032 and linux-azure 4.15.0-1074. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1867220 Title: Assignment of VDEV Somtimes Fails using Intel QAT To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1867220/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
So let me summarize your findings on the same host of yours (I suppose
your VMs use the same config for the number of vCPUs and the memory
size. I also suppose you only tested Hyper-V Generation 2 VMs or you
confirmed Gen-1 vs. Gen-2 makes no difference):
("fast" means you can see the GUI desktop or the text terminal prompt in
about 1~2 seconds, and "slow" means you need a much longer time, e.g. 1
minute (?))
fresh Server 19.10 ==> fast
fresh Server 19.10 + the ubuntu-desktop package ==> slow
fresh Desktop 19.10 ==> slow
fresh Desktop 19.04 ==> fast
fresh Desktop 19.04 upgraded to 19.10 ==> slow
So it looks a change in 19.10 with the xorg causes the slowness.
However, I can not reproduce the issue, because both my fresh 19.10 and
19.04 VMs boot up in 20+ seconds and I never have a boot-up time of 1~2
seconds.
Hi M, can you please check this case:
fresh Desktop 19.04 upgraded to 19.10 ==> slow
What if you boot the VM with the 19.04 kernel + 19.10's userspace (including
Xorg)?
If it's also slow, then we have more confidence that the 19.04 Xorg has an
issue.
If it's fast, then the issue may be more likely that the interaction between
the 19.10 Xorg and the 19.10 kernel is causing the issue.
Can you also please try booting the VM with a "good" 19.04 VM but (ONLY)
upgrading the kernel to 19.10?
In the slow cases, can you check the logs files (/var/log/Xorg*,
/var/log/syslog*) and see if there is any obvious error/warning?
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1848534
Title:
[Microsoft Hyper-V guest] System shows graphic artifacts for a moment,
then text cursor for about minute and then starts
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1848534/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Can Ubuntu devs please try to repro the issue? I can not repro it. :-( Hi M, I assume you can also repro the issue with a VM created from scratch from the server .iso (see comment #28) with a minimal installation? If yes, can you please share the vhdx file? If you configure the disk size to 15GB an use xfs (rather than ext4) in the installation process, the generated vhdx file should be 1.5GB or so (IIRC), so I guess there might be a way for you to share the file somewhere for me to download? Please also use less CPUs (e.g. 2) and memory for the VM (e.g. 2GB), if this doesn't prevent you from reproducing the issue. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
The typical boot-up time of my Ubuntu VM on Hyper-V is 20~30 seconds for a Desktop version of Ubuntu, and 10~20 seconds for a Server version. I tried Ubuntu 19.04 just now and it also took 20+ seconds. I never achieve a boot-up time of 2s. I do know Ubuntu can boot up fast in 2~3 seconds in WSL (Windows Subsystem for Linux), though I didn't try it in person. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Hi M, since I can not reproduce the delay issue, I don't know what I can do now. :-( Do you think if it's related to Xorg? Can you install a new VM from scratch from the server .iso (http://releases.ubuntu.com/19.10/ubuntu-19.10-live-server-amd64.iso) and see if you can reproduce the same issue? The server .iso doesn't install Xorg, and a lot of other packages used in a Desktop environment. I hope you can not repro the issue with it, then we'll have a good starting point. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
I'm not sure what exact issues you're reporting. Your VM takes too much time to boot up? How long? "systemd-analyze blame" should collect the info for your VM. Your VM's screen is somehow messed up temporarily during the boot-up process? Or the boot-up process is stuck in the "text cursor" screen for a long period of time (about 1 minute?) and you'd like to figure out what's happening during that period of time? But since it looks your VM is able to boot up in 2 minutes or so (?), it looks there is no fatal issue? You're saying you can reproduce the issue with a fresh VM created from the .iso file (ubuntu-19.10-desktop-amd64.iso)? It would be helpful if you can share your output of the same commands I ran. A video of your VM's boot-up process would be helpful, if that's not difficult. I don't know how you would share the video. :-) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1848534] Re: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts
Hi M, I can not reproduce the issue: just now I downloaded http://dt0cinyuc0rrg.cloudfront.net/ubuntu-19.10-desktop-amd64.iso and created a Generation-2 VM on Hyper-V with the .iso file. The VM boots up fast: it boots up to the Xorg desktop in 28 seconds with 1 CPU and 2GB memory, and in 21 seconds with 8 CPUs and 8GB memory. I don't see anything unusual. I believe you're also using Generation-2 VM since your "lspci" returns nothing. FYI: I got the below in my VM: decui@decui-u1910-gen2:~$ uname -a Linux decui-u1910-gen2 5.0.0-32-generic #34-Ubuntu SMP Wed Oct 2 02:06:48 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux decui@decui-u1910-gen2:~$ dmesg | grep "Hyper-V Host Build" [0.00] Hyper-V Host Build:18928-10.0-1-0.1044 decui@decui-u1910-gen2:~$ systemd-analyze Startup finished in 303ms (firmware) + 663ms (loader) + 1.087s (kernel) + 35.041s (userspace) = 37.096s graphical.target reached after 34.963s in userspace decui@decui-u1910-gen2:~$ systemd-analyze blame 31.530s plymouth-quit-wait.service 10.598s gdm.service 1.817s dev-sda2.device 944ms networkd-dispatcher.service 866ms NetworkManager-wait-online.service . decui@decui-u1910-gen2:~$ systemd-analyze critical-chain The time after the unit is active or started is printed after the "@" character. The time the unit takes to start is printed after the "+" character. graphical.target @34.963s └─multi-user.target @34.963s └─kerneloops.service @4.213s +34ms └─network-online.target @4.170s └─NetworkManager-wait-online.service @3.301s +866ms └─NetworkManager.service @2.983s +317ms └─dbus.service @2.912s └─basic.target @2.765s └─sockets.target @2.765s └─snapd.socket @2.754s +10ms └─sysinit.target @2.731s └─apparmor.service @2.199s +530ms └─local-fs.target @2.173s └─run-user-1000-gvfs.mount @17.395s └─run-user-1000.mount @14.119s └─local-fs-pre.target @456ms └─keyboard-setup.service @223ms +232ms └─systemd-journald.socket @218ms └─-.mount @206ms └─system.slice @206ms └─-.slice @206ms -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1848534 Title: [Microsoft Hyper-V guest] System shows graphic artifacts for a moment, then text cursor for about minute and then starts To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1848534/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1822133] Re: Azure Instance never recovered during series of instance reboots.
@Vald: This is from your attachment: [21965.367843] kernel BUG at /build/linux-azure-njdnVX/linux-azure-4.15.0/net/ipv4/ip_output.c:636! [21965.377590] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.15.0-1056-azure #61-Ubuntu [21965.435777] RIP: 0010:ip_do_fragment+0x571/0x860 [21965.435777] ip_fragment.constprop.47+0x43/0x90 [21965.435777] ip_finish_output+0xf6/0x270 [21965.435777] ip_output+0x75/0xf0 [21965.435777] ip_forward_finish+0x51/0x80 [21965.435777] ip_forward+0x38a/0x480 [21965.435777] ip_rcv_finish+0x122/0x410 [21965.435777] ip_rcv+0x292/0x360 [21965.435777] __netif_receive_skb_core+0x809/0xbc0 [21965.435777] __netif_receive_skb+0x18/0x60 [21965.435777] netif_receive_skb_internal+0x37/0xe0 [21965.435777] napi_gro_receive+0xd0/0xf0 [21965.435777] netvsc_recv_callback+0x16d/0x220 [hv_netvsc] [21965.435777] rndis_filter_receive+0x23b/0x580 [hv_netvsc] [21965.435777] netvsc_poll+0x17e/0x630 [hv_netvsc] [21965.435777] net_rx_action+0x265/0x3b0 [21965.435777] __do_softirq+0xf5/0x2a8 [21965.435777] irq_exit+0x106/0x110 [21965.435777] hyperv_vector_handler+0x63/0x76 So you're running a 4.15.0-1056-azure kernel, which panics at RIP: 0010:ip_do_fragment+0x571/0x860 (net/ipv4/ip_output.c:636). This is a known issue in this version of the kernel and we're working on a new version which will fix the issue: The identified upsream fix is: commit 5d407b071dc369c26a38398326ee2be53651cfe4 Author: Taehee Yoo Date: Mon Sep 10 02:47:05 2018 +0900 Subject: ip: frags: fix crash in ip_do_fragment() ( https://github.com/torvalds/linux/commit/5d407b071dc369c26a38398326ee2be53651cfe4 ) Meanwhile, if it's possible, please downgrade the kernel to from -1056 to -1052, which per support does not crash. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1822133 Title: Azure Instance never recovered during series of instance reboots. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1822133/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1837661] Re: [linux-azure] CRI-RDOS | Live migration only takes 10 seconds, but the VM was unavailable for 2 hours
I guess you might have already included this patch:
15becc2b56c6 ("PCI: hv: Add hv_pci_remove_slots() when we unload the
driver")
Unluckily it turns out it is buggy and just now I had to post a further patch
for it:
[PATCH] PCI: hv: Fix panic by calling hv_pci_remove_slots() earlier
( https://lkml.org/lkml/2019/8/1/1173)
Please consider including this further patch as well.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1837661
Title:
[linux-azure] CRI-RDOS | Live migration only takes 10 seconds, but the
VM was unavailable for 2 hours
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1837661/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826416] Re: [Xenial] Customer can not SSH to Linux VM due to "VSC State Unhealthy"
I can confirm the fix is included in the kernel "4.4.0-152.179": https://kernel.ubuntu.com/git/ubuntu/ubuntu- xenial.git/tree/include/linux/hyperv.h?h=Ubuntu-4.4.0-152.179 I installed the kernel, did some network tests, and it worked fine for me: #apt-get install linux-image-4.4.0-152-generic #reboot # uname -a Linux localhost 4.4.0-152-generic #179-Ubuntu SMP Thu Jun 13 10:05:07 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux ** Tags removed: verification-needed-xenial ** Tags added: verification-done-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826416 Title: [Xenial] Customer can not SSH to Linux VM due to "VSC State Unhealthy" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1826416/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1822133] Re: Azure Instance never recovered during series of instance reboots.
@rnsc: Can you please share the VM's full serial log, which can be obtained from Azure portal's Boot Diagnostics -> Serial log? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1822133 Title: Azure Instance never recovered during series of instance reboots. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1822133/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1822133] Re: Azure Instance never recovered during series of instance reboots.
I tried to reproduce the bug but couldn't. My Ubuntu 18.04 VM (in West US 2, the VM size is: Basic A3 (4 vcpus, 7 GiB memory)) is still running fine, after I rebooted the VM 100 times with the below /etc/rc.local script: #!/bin/bash date >> /root/reboot.log NUM=`wc -l /root/reboot.log | cut -d' ' -f1` [ $NUM -le 100 ] && reboot In Kirk's log, it looks the VM failed to obtain an IP via DHCP (or somehow the VM's network went DOWN?), and there are a lot of lines of WARNING ExtHandler CGroup walinuxagent.service: Crossed the Memory Threshold. Current Value: 627945472 bytes, Threshold: 512 megabytes. I don't know what the line means. I don't see any issue in Linux kernel and drivers. I guess the issue may be in the waagent/cloud-init daemons, or somewhere outside the VM. Since I can not reproduce the issue, and it looks nobody can reproduce the issue now (it looks Kirk's VM works fine now, after the VM was Stopped and Started again), I can not further debug the issue. If somebody manages to repro the issue again, please check if you're still able to login the VM via Azure serial console; if yes, please check if the VM has an IP address by "ifconfig -a"; if the VM has no IP, we'll have to check the syslog (and/or the network manager's log?) to find out what goes wrong; if the VM has an IP but is unable to communicate to the outside world, something may be wrong with the Linux network device driver. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1822133 Title: Azure Instance never recovered during series of instance reboots. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1822133/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1822133] Re: Azure Instance never recovered during series of instance reboots.
@Kirk: I suppose you can get your VM back by Restarting the VM by force via Azure Portal (or Azure cmd line)? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1822133 Title: Azure Instance never recovered during series of instance reboots. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1822133/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1822133] Re: Azure Instance never recovered during series of instance reboots.
@Kirk: Can you please share the VM's serial log, which can be obtained from Azure portal's Boot Diagnostics -> Serial log? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1822133 Title: Azure Instance never recovered during series of instance reboots. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1822133/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1822133] Re: Azure Instance never recovered during series of instance reboots.
It's glad to see the issue can not repro with the 5.0.0-1007.7 kernel. @sfeole: The line "[ 84.247704]hyperv_fb: unable to send packet via vmbus" usually means the VM has panicked. Do you happen to still keep the full serial log containing this line of error? It would be good to understand this error for the 4.18.0-1013-azure kernel. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1822133 Title: Azure Instance never recovered during series of instance reboots. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1822133/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826416] Re: [Xenial] Customer can not SSH to Linux VM due to "VSC State Unhealthy"
When the issue happens, there is no error message in dmesg or syslog, and it's just the host side NetVSP driver stops reading from the guest- to-host ring, and the guest network stops working. So we don't really have any logs to provide here. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826416 Title: [Xenial] Customer can not SSH to Linux VM due to "VSC State Unhealthy" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1826416/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826453] Re: Grub2 Booting in blind mode due to "invalid video mode specification `text'"
Hi Mathieu, Thanks for the hint! After I changed the Pin-Priority from 400 to 600 in the below file, I successfully installed the *3.22 version: root@decui-g2-1604:~# cat /etc/apt/preferences.d/proposed-updates Package: * Pin: release a=xenial-proposed Pin-Priority: 600 ... root@decui-g2-1604:~# apt policy grub2-common grub2-common: Installed: 2.02~beta2-36ubuntu3.22 Candidate: 2.02~beta2-36ubuntu3.22 Version table: *** 2.02~beta2-36ubuntu3.22 600 600 http://archive.ubuntu.com/ubuntu xenial-proposed/main amd64 Packages 100 /var/lib/dpkg/status 2.02~beta2-36ubuntu3.21 500 500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages 2.02~beta2-36ubuntu3 500 500 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages And I can confirm the issue is fixed for "16.04.6 LTS (Xenial Xerus)"! Now in the VM running on Hyper-V, I get the below as expected (previously I can not see the line with the *3.21 version of grub2): root@decui-g2-1604:~# dmesg | grep "efifb: framebuffer" [0.472000] efifb: framebuffer at 0xf800, mapped to 0xc90001c0, using 3072k, total 3072k Thank you all for the quick response! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826453 Title: Grub2 Booting in blind mode due to "invalid video mode specification `text'" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1826453/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826453] Re: Grub2 Booting in blind mode due to "invalid video mode specification `text'"
Hi Brian, "apt-get update; apt-get install grub2-common" still reports "grub2-common is already the newest version (2.02~beta2-36ubuntu3.21)" to me, and the new version can not be found. I'm running the Ubuntu 16.04.6 LTS (4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64) version and I have "deb http://archive.ubuntu.com/ubuntu/ xenial-proposed restricted main multiverse universe" in my /etc/apt/sources.list. I guess I must be missing something? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826453 Title: Grub2 Booting in blind mode due to "invalid video mode specification `text'" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1826453/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826453] Re: Grub2 Booting in blind mode due to "invalid video mode specification `text'"
I added deb http://archive.ubuntu.com/ubuntu/ xenial-proposed restricted main multiverse universe into /etc/apt/sources.list and ran "apt-get update", and when I tried to upgrade to the .22 version, it can not be found: root@decui-g2-1604:~# apt-get install grub2-common Reading package lists... Done Building dependency tree Reading state information... Done grub2-common is already the newest version (2.02~beta2-36ubuntu3.21). grub2-common set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 70 not upgraded. root@decui-g2-1604:~# apt-get install grub2-common=2.02~beta2-36ubuntu3.22 Reading package lists... Done Building dependency tree Reading state information... Done E: Version '2.02~beta2-36ubuntu3.22' for 'grub2-common' was not found -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826453 Title: Grub2 Booting in blind mode due to "invalid video mode specification `text'" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1826453/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826453] Re: Grub2 Booting in blind mode due to "invalid video mode specification `text'"
It looks the .22 binary (2.02~beta2-36ubuntu3.22) for amd64 has not appeared in the -proposed repo yet. I can not find it in https://launchpad.net/ubuntu/xenial/+package/grub- common and http://archive.ubuntu.com/ubuntu/dists/xenial- proposed/main/uefi/grub2-amd64/ (please let me know if I'm looking at the wrong place) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826453 Title: Grub2 Booting in blind mode due to "invalid video mode specification `text'" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1826453/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826453] Re: Grub2 Booting in blind mode due to "invalid video mode specification `text'"
Hi Mathieu, thank you for the explanation and pushing the fix! Hi Brian, I'll test the new grub2 binary in a Ubuntu 16.04.6 VM on Hyper-V. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826453 Title: Grub2 Booting in blind mode due to "invalid video mode specification `text'" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1826453/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826453] Re: Grub2 Booting in blind mode due to "invalid video mode specification `text'"
I was not clear about the lifecycle of 16.04 and 14.04. :-) AFAIK, there is no real need to fix the bug for 14.04. And I understand it looks there won't be a 16.04.7, either. Since the bug is already fixed in 18.04 (and newer? I didn't check 19.xx), it looks to me we can leave it as is for 16.04 and 14.04, and mark the bug as WON'T FIX, if there is nothing we can really do for 16.04 and 14.04. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826453 Title: Grub2 Booting in blind mode due to "invalid video mode specification `text'" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1826453/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1826453] [NEW] Grub2 Booting in blind mode due to "invalid video mode specification `text'"
Public bug reported: In a Ubuntu 16.04.6 VM, which runs as a Gen2 VM on Hyper-V, I add GRUB_GFXPAYLOAD_LINUX="text" into /etc/default/grub and run update-grub; next, grub2 prints the below error: error: invalid video mode specification `text'. Booting in blind mode This means grub2 passes a zero value for the "lfb_base" to Linux kernel, which then fails to reserve the framebuffer MMIO range in drivers/hv/vmbus_drv.c: vmbus_reserve_fb(); as a result, when we pass through a PCIe device to the VM, the PCIe device may get a PCI MMIO BAR in the FB MMIO range, causing a conflict, and the PCIe device can not work in the VM. The issue can not reproduce with Ubuntu 18.04. It turns out the grub2 in Ubuntu 18.04 has the below fix: video: skip 'text' gfxpayload if not supported, to fallback to default (https://git.launchpad.net/~ubuntu-core-dev/grub/+git/ubuntu/commit/?id=38d9e2f862a96a039ae8ca8b87b8615d154ceda4). I'm asking the patch author (Mathieu Trudel-Lapierre) to submit the patch to the upstream grub and this is the upstream bug link: https://savannah.gnu.org/bugs/?56217 . Here this launchpad bug is for Ubuntu 16.04 and 14.04. ** Affects: grub2 (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1826453 Title: Grub2 Booting in blind mode due to "invalid video mode specification `text'" To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1826453/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1805304] Re: [Hyper-V] Additional patches for Lv2 storage performance
The link to "[PATCH] scsi: storvsc: Fix a race in sub-channel creation that can cause panic" is https://lkml.org/lkml/2018/11/26/159 or https://lore.kernel.org/patchwork/patch/1016903/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1805304 Title: [Hyper-V] Additional patches for Lv2 storage performance To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1805304/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792349] Re: Memory leaking when running kubernetes cronjobs
More patches are required: https://lkml.org/lkml/2018/11/2/182 It looks we'll have to wait for some time, before the kernel stabilizes... -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792349 Title: Memory leaking when running kubernetes cronjobs To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1792349/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1777128] Re: [Hyper-V] patches for SR-IOV post-bionic GA
I guess we can close the bug now? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1777128 Title: [Hyper-V] patches for SR-IOV post-bionic GA To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1777128/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1786313] Re: [Hyper-V] hv_netvsc: Fix napi reschedule while receive completion is busy
Thanks, Marcolo! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1786313 Title: [Hyper-V] hv_netvsc: Fix napi reschedule while receive completion is busy To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1786313/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1786313] Re: [Hyper-V] hv_netvsc: Fix napi reschedule while receive completion is busy
4.15.0-1022 (https://git.launchpad.net/~canonical-kernel/ubuntu/+source /linux-azure/tree/drivers/net/hyperv/netvsc.c?h=master-next=Ubuntu- azure-4.15.0-1022.22_16.04.1) does NOT have the fix (6b81b193b83e87da1ea13217d684b54fccf8ee8a). I'm not sure why the bug status was changed to Fix Committed on Aug 17. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1786313 Title: [Hyper-V] hv_netvsc: Fix napi reschedule while receive completion is busy To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1786313/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1747970] Re: [Hyper-V] vsock: always call vsock_init_tables()
Yes, I confirm this is the patch we need for the Enhanced Session mode.
The patch was originally made to fix a different issue, but it can also
fix the VM panic we see with Enhanced Session mode, because the cause is
the same: there is a race
static int __init hvs_init(void)
{
int ret;
if (vmbus_proto_version < VERSION_WIN10)
return -ENODEV;
ret = vmbus_driver_register(_drv);
if (ret != 0)
return ret;
Dexuan: ---: if the host-initiated connection comes
here (e.g. before we call vsock_core_init(), hvs_open_connection() ->
vsock_find_bound_socket() -> __vsock_find_bound_socket() can access the
uninitialized vsock_bound_sockets(addr) -> vsock_bind_table, which will be
initialized in vsock_core_init() -> vsock_init_tables(), and we'll hit a panic.
ret = vsock_core_init(_transport);
if (ret) {
vmbus_driver_unregister(_drv);
return ret;
}
return 0;
}
The latest upstream 4.15.1 kernel has the fix already, but 4.14.17 doesn't
have the fix.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1747970
Title:
[Hyper-V] vsock: always call vsock_init_tables()
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1747970/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1745169] Re: Kernel tried to execute NX-protected page - exploit attempt?
Is this a 100% repro? If yes, can you check if adding the "nopti" kernel parameter can fix the issue? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1745169 Title: Kernel tried to execute NX-protected page - exploit attempt? To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1745169/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1735546] Re: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1
@mhcerri Thanks for the explanation! I built the kernels directly from the git repo and I didn't use the binary packages (I thought they should be the same). What's your repro-rate? Yesterday as Chris tested linux-azure-edge 4.13.0-1004.4 + my pull request, the repro rate was only ~1%, meaning we have to do "unload/reload netvsc" several hundred times to repro the hang or the calltrace. Previously without the pull request, Chris mentioned it was much easier to repro the issue. So it looks to me the bug is timing-sensitive, and somehow my pull request made it difficult to repro it. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1735546 Title: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1735546/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1735546] Re: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1
@mhcerri FYI: I'm not using any kernel from http://kernel.ubuntu.com/~mhcerri/azure/. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1735546 Title: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1735546/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1735546] Re: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1
@mhcerri I'm using https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-azure/, e.g. for 4.13.0-1004-azure-edge, I mean: https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-azure/commit/?h=azure-edge-next=21d8a99f88af972684618521cf19adafe24dc566. Please let me know in case I'm using the incorrect repo? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1735546 Title: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1735546/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1735546] Re: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1
@mhcerri I only tested Gen1 VM (4.13.0-1004-azure-edge) on my local Hyper-V host (WS 2016), and couldn't repro the issue, i.e. reloading hv_netvsc and changing MTU worked fine. As I understand, I think the bug was originally reported against 4.13.0-1001.1 (I did not test this version). That's why I think somehow the bug was fixed in 4.13.0-1004-azure-edge. Ubuntu-azure-edge-4.13.0-1001.1 is 567ef14ee13c5c4e336121106cc19733800d618e, and Ubuntu-azure-edge-4.13.0-1004.4 is 21d8a99f88af972684618521cf19adafe24dc566. There are non-trivial changes between them: root@decui-1604:/opt/linux-azure# git diff 567ef14ee13c5c4e336121106cc19733800d618e 21d8a99f88af972684618521cf19adafe24dc566 -- drivers/hv/ drivers/net/hyperv/| wc -l 2353 root@decui-1604:/opt/linux-azure# git diff 567ef14ee13c5c4e336121106cc19733800d618e 21d8a99f88af972684618521cf19adafe24dc566 -- drivers/hv/ drivers/net/hyperv/| grep ^diff diff --git a/drivers/hv/Kconfig b/drivers/hv/Kconfig diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c diff --git a/drivers/hv/hv_fcopy.c b/drivers/hv/hv_fcopy.c diff --git a/drivers/hv/hv_kvp.c b/drivers/hv/hv_kvp.c diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c root@decui-1604:/opt/linux-azure# git diff 567ef14ee13c5c4e336121106cc19733800d618e 21d8a99f88af972684618521cf19adafe24dc566 -- drivers/hv/ drivers/net/hyperv/| wc -l 2353 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1735546 Title: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1735546/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1735546] Re: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1
@leann Actually I'm not sure if the pull request in bug 1736283 can fix this bug. It looks 4.13.0-1004-azure-edge has already fixed this bug somehow. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1735546 Title: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1735546/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1736283] Re: linux-azure-edge: [Ubuntu-azure-edge-4.13.0-1005.5]: refresh the rescind-handling, hv_sock and vPCI drivers
BTW, the 19 commits in the pull request is on this branch: https://github.com/dcui/linux/commits/decui/azure-edge-next-Ubuntu-azure-edge-4.13.0-1005.5-20171204 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736283 Title: linux-azure-edge: [Ubuntu-azure-edge-4.13.0-1005.5]: refresh the rescind-handling, hv_sock and vPCI drivers To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1736283/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1736283] Re: linux-azure-edge: [Ubuntu-azure-edge-4.13.0-1005.5]: refresh the rescind-handling, hv_sock and vPCI drivers
To resolve the above 3 issues, I created this pull request based on Ubuntu-azure-edge-4.13.0-1005.5 (https://git.launchpad.net/~canonical- kernel/ubuntu/+source/linux-azure/commit/?h=azure-edge- next=ec62f77bbe0697ce128f71fac4fc45c99b6f40d1). The pull request is hosted in my own git repo: https://github.com/dcui/linux/compare/ec62f77bbe0697ce128f71fac4fc45c99b6f40d1...dcui:decui/azure-edge-next-Ubuntu-azure-edge-4.13.0-1005.5-20171204 The pull request consists of the below changes: 1) I reverted the old version of hv_sock driver and cherry-picked the upstream version of the driver, and the related bug fixes in hv_sock and vmbus drivers; 2) To further fix the rescind-handling, I cherry-picked a patch from KY, and I made the patch " UBUNTU: SAUCE: vmbus: unregister device_obj->channels_kset" (which hasn't been in any upstream repo as of today). With the 2 patches, issue #1 and #2 are fixed. 3) I cherry-pick "PCI: hv: Use effective affinity mask" to fix the issue #3. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736283 Title: linux-azure-edge: [Ubuntu-azure-edge-4.13.0-1005.5]: refresh the rescind-handling, hv_sock and vPCI drivers To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1736283/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1736283] [NEW] linux-azure-edge: [Ubuntu-azure-edge-4.13.0-1005.5]: refresh the rescind-handling, hv_sock and vPCI drivers
Public bug reported: Ubuntu-azure-edge-4.13.0-1005.5 (https://git.launchpad.net/~canonical- kernel/ubuntu/+source/linux-azure/tag/?h=azure-edge-next=Ubuntu- azure-edge-4.13.0-1005.5) has some bugs: 1) After "Disable and re-Enable the Integration Services devices (Time Sync, Hearbeat, Shutdown, etc)", the devices can't come back. 2) For a VM with SR-IOV VF configured, the PCI VF device can't come back after we Pause and Resume the VM. 3) When we assign 7 Mellanox ConnectX-3 VFs to a 32-vCPU VM, one of the VFs may fail to receive interrupts, and the Vf driver will time out and fail to create the 7th VF network interface. My VM info: # lsb_release -rd Description:Ubuntu 16.04.3 LTS Release:16.04 (I'm using the Ubuntu-azure-edge-4.13.0-1005.5 kernel.) ** Affects: linux-azure (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1736283 Title: linux-azure-edge: [Ubuntu-azure-edge-4.13.0-1005.5]: refresh the rescind-handling, hv_sock and vPCI drivers To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1736283/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1735546] Re: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1
BTW, I tested "modprobe -r hv_netvsc; modprobe hv_netvsc" and "ip link set dev eth0 mtu 3000; ip link set dev eth0 mtu 4500" with 4.13.0-1004 -azure-edge on WS 2016 (Version 1607, OS build 14393:1943). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1735546 Title: [hyper-v] reloading netvsc issue on linux-azure 4.13.0-1001.1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux-azure/+bug/1735546/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
