[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-08 Thread Dan Streetman
sigh, well that illustrates why this design was a bad idea. I changed the param to use their short form which should keep the service filename length under the limit. To be totally clear, this is in no way a correct fix, I just want to confirm this is actually the problem; then it'll have to get

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-08 Thread Jennifer Duong
Dan, I've attached /var/log/syslog and journalctl logs of a recreate after installing nvme-cli_1.9-1ubuntu0.1+bug1874270v20210408b2_amd64 and rebooting the host. Apr 8 14:44:00 ICTM1608S01H1 root: JD: Resetting controller B Apr 8 14:44:09 ICTM1608S01H1 kernel: [ 196.190003] lpfc :af:00.1:

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-08 Thread Dan Streetman
> It looks like connect-all didn't recognize the "matching" flag ah sorry, that param is used in later versions, i uploaded a new buidl to the ppa can you try that one? It just started building now, so might take some time to finish building and get published. ** Changed in: nvme-cli (Ubuntu)

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-08 Thread Jennifer Duong
Dan, I've attached /var/log/syslog and journalctl logs of a recreate after installing nvme-cli_1.9-1ubuntu0.1+bug1874270v20210408b1_amd64 and rebooting the host. It looks like connect-all didn't recognize the "matching" flag. Apr 8 11:48:45 ICTM1608S01H1 root: JD: Resetting controller B Apr 8

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-08 Thread Dan Streetman
@jduong can you test with the nvme-cli package from this ppa: https://launchpad.net/~ddstreet/+archive/ubuntu/lp1874270 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1874270 Title: NVMe/FC

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-07 Thread Dan Streetman
Hmm, I looked at the udev rule and the service it calls a bit closer, and I'm not sure if that udev rule has *ever* worked; it's attempting to pass a full list of parameters as the template value on the cmdline, and I can kind of understand the intention, but it's not a good way to implement it.

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-07 Thread Jennifer Duong
Dan, where do I change the kernel rport timeout? And how can I go about changing the timeout on a server with Emulex cards installed versus Qlogic? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-07 Thread Dan Streetman
It looks like the service is failing because your controller is in the process of resetting, which appears to take several minutes. I'm not sure what the design is for nvme-cli tools handling such a long reset time, but my first guess would be to increase the kernel rport timeout, which appears to

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-07 Thread Jennifer Duong
At the time in which a storage controller is failed, /var/log/syslog and journalctl look identical: Apr 7 11:45:28 ICTM1608S01H1 kernel: [586649.657080] lpfc :af:00.1: 5:(0):6172 NVME rescanned DID x3d0a00 port_state x2 Apr 7 11:45:28 ICTM1608S01H1 kernel: [586649.657268] lpfc

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-07 Thread Dan Streetman
> Dan, where can I find the location of these logs? they should be in /var/log/syslog, and/or the journal, which you can check with: for all logs: $ journalctl just logs for this boot: $ journalctl -b just logs for the nvmf-connect unit(s): $ journalctl -u 'nvmf-connect*' You probably only

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-04-05 Thread Jennifer Duong
Dan, where can I find the location of these logs? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1874270 Title: NVMe/FC connections fail to reestablish after controller is reset To manage

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-03-31 Thread Matt Schulte
** Changed in: nvme-cli (Ubuntu) Status: Expired => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1874270 Title: NVMe/FC connections fail to reestablish after controller is reset

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2021-01-05 Thread Launchpad Bug Tracker
[Expired for nvme-cli (Ubuntu) because there has been no activity for 60 days.] ** Changed in: nvme-cli (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2020-11-06 Thread Dan Streetman
You'll need to check the specific output/log of the nvmf-connect@ services that are failing, e.g.: Apr 21 16:48:53 ICTM1610S01H2 systemd-udevd[2946]: fc_udev_device: Process 'systemctl --no-block start nvmf-connect@--device=none\t--

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2020-10-21 Thread Launchpad Bug Tracker
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: nvme-cli (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1874270 Title:

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2020-05-22 Thread Jennifer Duong
I am still seeing this with Ubuntu 20.04 LTS -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1874270 Title: NVMe/FC connections fail to reestablish after controller is reset To manage notifications

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2020-04-22 Thread Jennifer Duong
Also, it does not look like Broadcom's website has an autoconnect script that supports Ubuntu. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1874270 Title: NVMe/FC connections fail to reestablish

[Bug 1874270] Re: NVMe/FC connections fail to reestablish after controller is reset

2020-04-22 Thread Jennifer Duong
I've attached logs. ** Attachment added: "ICTM1610S01-messages-4-21-20.zip" https://bugs.launchpad.net/ubuntu/+source/nvme-cli/+bug/1874270/+attachment/5357988/+files/ICTM1610S01-messages-4-21-20.zip -- You received this bug notification because you are a member of Ubuntu Bugs, which is