[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-30 Thread Norbert B.
Update: Problem is still present in: Ubuntu 20.04.4 LTS Kernel: Linux 5.13.0-1030-oracle OCFS2-Modules were gone after upgrade and had to be reinstalled with apt install linux-modules-extra-5.13.0-1030-oracle Problem is (after current upgrades) still NOT present in: Ubuntu 18.04.6 LTS Kernel: L

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-16 Thread Mauricio Faria de Oliveira
Last week has been a bit busy and I couldn't get to this, sorry; it's still on my list. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1959581 Title: OCFS2 intermittently not mountable on a second No

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-05 Thread Norbert B.
Hello Mauricio, thank you for picking up the problem. I confirm your understanding and attach setup steps and logs. I can give you more details if needed - just ask me. I'm in office from Monday to Thursday. Best regards ** Attachment added: "setup steps.txt" https://bugs.launchpad.net/ubun

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-05 Thread Mauricio Faria de Oliveira
Hi Norbert, This would suggest this indeed isn't related to that kernel commit (different symptom: hang vs. crash; and independently of quotas). I'd like to check this in more detail on the kernel side, next week. Could you please confirm whether my understanding to reproduce this is correct? -

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-05 Thread Norbert B.
I hope, that this will work. What irritates me: I did tests with deactivated quotas on system and disk, but the problem remained. We don't have the problem with Oracle Linux (RHEL), which uses regular kernels too, afaik. -- You received this bug notification because you are a member of Ubuntu

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-04 Thread Mauricio Faria de Oliveira
Forgot this line, > Those 2 versions are pulled by our ubuntu 5.13 generic kernel stable > maintenance, > which is currently at v5.15.26 (so a few to go for v5.15.33). ~/git/kernel/ubuntu-impish$ git log --oneline origin/master-next | grep stable | head -n1 ffd44d75c03e UBUNTU: upstream stable

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-04 Thread Mauricio Faria de Oliveira
Hi, I couldn't look at this bug in more detail, but I checked kernel git repos. It's not yet in ubuntu generic/oracle 5.13 kernel, but it's coming sometime. ... The commit was introduced in linux mainline in v5.18, and is present in linux stable v5.15.33, but not yet in v5.10.x. ~/git/linux$ g

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-03 Thread Lucas Kanashiro
Thanks for getting back to us with more information Norbert. I went through the Debian bug and it seems to be a linux bug instead of ocfs2-tools one. The Debian maintainer came to a conclusion that this commit needs to be included in the linux build: From de19433423c7bedabbd4f9a25f7dbc62c5e78921

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-05-02 Thread Norbert B.
Unfortunately the problem is back in: Ubuntu 20.04.4 LTS Kernel: Linux 5.13.0-1027-oracle ocfs2-tools 1.8.6-2ubuntu1 open-iscsi 2.0.874-7.1ubuntu6.2 ... or was never solved an the solution before was just a lucky accident. Meanwhile I have two servers, who never can mount an OCFS2 volume at the s

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-03-08 Thread Lucas Kanashiro
Thanks for describing your findings Norbert, and good to hear you have a functional environment now. This might be helpful for others. In this case, I do not think there is much we can do here, since this is a very specific case in Oracle Cloud. -- You received this bug notification because you a

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-03-07 Thread Norbert B.
Hello Paride & all, sorry for the late reply but I was on vacation last week. I found a solution/workaround now but it's a little bit strange. Background: Oracle is offering different shapes of virtual machines. In the OCFS2 cluster we are using: VM.Standard.E4.Flex (AMD) VM.Standard3.Flex (Intel

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-28 Thread Norbert B.
Hello Paride, thank you for the detailed instruction - very helpful! I hope it will work in Oracle environment, because they have their own Oracle repositories. For the moment I'm testing with current updates and new kernel 5.13.0-1018-oracle. The error didn't happen immediately after the updat

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-28 Thread Paride Legovini
@Norbert, as Lucas suggested please try using `do-release-upgrade`. You'll have to edit /etc/update-manager/release-upgrades and set: Prompt=normal then just running `do-release-upgrade` should upgrade your system to Impish (21.10). If you can still reproduce the issue there, please also tr

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-22 Thread Lucas Kanashiro
I did not find any Oracle specific doc on how to upgrade a Ubuntu system to Impish, which is kind of expected since they do not support this use case. I'd recommend you to try to use `do-release-upgrade` to do that. -- You received this bug notification because you are a member of Ubuntu Bugs, wh

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-21 Thread Norbert B.
Hello Paride, thank you for the effort with the downgrade version. The problem stays the same in open-iscsi 2.0.874-5ubuntu2.10~focal2. Should we try a downgrade of ocfs2? Do you know a description somewhere, how to upgrade an Oracle Focal LTS version to Impish Indri? It would be helpful for me

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-18 Thread Paride Legovini
Hi Norbert, I had a look at the differences between those two versions but I found no clue. In any case I prepared a PPA with Bionc's open-iscsi compiled for Focal: https://launchpad.net/~paride/+archive/ubuntu/lp1959581 you can use this to verify that the issue isn't a regression that happened

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-16 Thread Norbert B.
Thanks @all for not leaving me alone with this issue! Open-iscsi versions: Ubuntu 20.04.3 LTS: 2.0.874-7.1ubuntu6.2 Ubuntu 18.04.6 LTS: 2.0.874-5ubuntu2.10 (Oracle Linux Server 7.9: 6.2.0.874-22) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribe

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-15 Thread Lucas Kanashiro
Norbert, Could you also provide which open-iscsi version you are using? I was checking the difference between open-iscsi in bionic and focal and the upstream version is the same but some patches were applied. For instance, this is one bug reported by one Orable Cloud user which was fixed in bionic

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-14 Thread Norbert B.
Hello Athos, here is the result: Ubuntu 20.04 Cluster 1: - Ubuntu 20.04 Cluster 2: - Ubuntu 18.04: mlocate 0.26-2ubuntu3.1 PRUNE_BIND_MOUNTS="yes" # PRUNENAMES=".git .bzr .hg .svn" PRUNEPATHS="/tmp /var/spool /media /var/lib/os-prober /var/lib/ceph /home/.ecryptfs /var/lib/schroot" PRUNEFS="NFS

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-10 Thread Athos Ribeiro
Hi Norbert, It would be nice if we'd be able to come up with a minimal reproducer here. in the meanwhile, before we discard the issue linked by Sergio, would you mind verifying in which cases (bionic, focal, and oracle linux) you do have locate (or mlocate) installed (dpkg -l | grep locate) and t

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-08 Thread Norbert B.
After Sergio didn't find any corresponding change in OCFS2 tools delta I could imagine, that there was a change in ISCSI adaption whicht could be the reason in combination with OCFS2? Unfortunately Oracle Cloud (OCI) does not offer Impish Indri as cloud image. They only support the LTS versions 18

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-07 Thread Utkarsh Gupta
Hi Norbert, Hm, interesting. I am not entirely sure what causes this (race?) condition given the kernel upgrade didn't fix this. But this is still not very clear where the problem is. As you noted, this is rather a very difficult problem to run into and requires the infra that I/we don't have acce

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-03 Thread Norbert B.
OK, the test with the new kernel went fast. The problem appeared at the first boot with the new kernel. Here some console output while trying to reboot: # @ first shutdown with new kernel [**] A stop job is running for Login to …SCSI targets (3min 36s / 4min 30s) [ 605.284682] INFO: task mou

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-03 Thread Norbert B.
Hello Sergio, thank you for picking up this bug. I realize that this is a very difficult problem. Regarding the possibilty of an upstream bug: Meanwhile there is a new kernel available: Linux 5.11.0-1028-oracle I will install it and do tests again. This can need some days, because the bug does n

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-02 Thread Sergio Durigan Junior
Thanks for the bug report, Norbert. First of all, I noticed that the ocfs2-tools package contains a delta relative to the Debian package. The delta is trivial, though, and is there just to make sure the package is built only on support architectures (see bug #1745155 for more details). I don't t

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-02-01 Thread Norbert B.
** Description changed: Environment: Oracle Cloud Infrastructure (OCI) 2 Nodes + ISCSI block volume with OCFS2 Operating System: Ubuntu 20.04.3 LTS Kernel: Linux 5.11.0-1027-oracle ocfs2-tools: 1.8.6-2ubuntu1 Problem: OCFS2 is intermittently not mountable on a second Node.

[Bug 1959581] Re: OCFS2 intermittently not mountable on a second Node in Ubuntu 20.04.3 LTS

2022-01-31 Thread Norbert B.
** Description changed: Environment: Oracle Cloud Infrastructure (OCI) 2 Nodes + ISCSI block volume with OCFS2 Operating System: Ubuntu 20.04.3 LTS Kernel: Linux 5.11.0-1027-oracle ocfs2-tools: 1.8.6-2ubuntu1 Problem: OCFS2 is intermittently not mountable on a second Node.