[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
This bug is believed to be fixed in curtin in version 18.2. If this is still a problem for you, please make a comment and set the state back to New Thank you. ** Changed in: curtin Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
** Merge proposal linked: https://code.launchpad.net/~raharper/curtin/+git/curtin/+merge/360217 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
This bug is fixed with commit ea939ee7 to curtin on branch master. To view that commit see the following URL: https://git.launchpad.net/curtin/commit/?id=ea939ee7 ** Changed in: curtin Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
** Merge proposal unlinked: https://code.launchpad.net/~raharper/curtin/+git/curtin/+merge/358104 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
My theory on different versions of GRUB causing the issue didn't pan out. After wiping the PReP partition I was able to go between 14.04, 16.04, and 18.04 with no problems. The only way I've been able to reproduce the issue after fixing it is by filling the PReP partition with garbage using $ dd if=/dev/urandom of=/dev/sda1 This may of happened in our CI by running a destructive storage test (badblock-destructive, fio). @Scott MAAS is already telling Curtin to wipe the PReP partition, Curtin isn't fully wiping it which is why we're seeing the problem. I was merely suggesting wiping PReP by default may be useful in non-MAAS environments to prevent this error. I see why you wouldn't want to do that so feel free to ignore my suggestion :) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
** No longer affects: maas -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
** Changed in: curtin Status: Confirmed => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
** Merge proposal linked: https://code.launchpad.net/~raharper/curtin/+git/curtin/+merge/358104 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
On Wed, Oct 31, 2018 at 9:20 AM Scott Moser wrote: > > With regard to wiping all prep partitions... > We do have code that does this in helpers/pt_prep. Ie, the non-storage > config path does this. > However, this is kind of a grey line here. If maas wants a partition wiped, > then MAAS should send config that says it should be wiped. > > If we bake in behavior to curtin of "Wipe all created (or existing?) PReP > partitions", then > a.) that differs in behavior from all other partition types, causing > inconsistency > b.) What happens at some later point when we for some reason need to *not* > wipe those (dual boot or some other path). Curtin is then left in this > unfortunate position of figuring out what MAAS wanted. > > Why can't we have MAAS wipe this partition? MAAS is sending wipe: zero for the prep partition, which is exactly what we want. I think you're right that we want MAAS to keep doing this and have curtin just follow the config. > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1800153 > > Title: > [2.5] Failed to deploy ppc64el when partition table is GPT > > To manage notifications about this bug go to: > https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
With regard to wiping all prep partitions... We do have code that does this in helpers/pt_prep. Ie, the non-storage config path does this. However, this is kind of a grey line here. If maas wants a partition wiped, then MAAS should send config that says it should be wiped. If we bake in behavior to curtin of "Wipe all created (or existing?) PReP partitions", then a.) that differs in behavior from all other partition types, causing inconsistency b.) What happens at some later point when we for some reason need to *not* wipe those (dual boot or some other path). Curtin is then left in this unfortunate position of figuring out what MAAS wanted. Why can't we have MAAS wipe this partition? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
On Tue, Oct 30, 2018 at 7:35 PM Lee Trager wrote: > > Every time I commission the PPC host we have in our CI I always get a > GPT partitioning table using MAAS 2.4.2+. Looking through the source > code it appears GPT is always set when creating a new partitioning > table[1] and when generating the preseed[2] due to the bios_boot_method > being powernv. I'm not sure how msdos got set but I believe the > partition table type is a red herring. > > The issue appears to be that Curtin is not wiping the PReP > partition(/dev/sda1). If I chroot into the deployment environment and > try to install GRUB with Well, other than simple mode, custom storage mode has never wiped a prep partition. while that's a bug, it doesn't explain why we only see this issue now. > > grub-install --target=powerpc-ieee1275 /dev/sda1 --no-nvram > > it fails. If I wipe /dev/sda1 first GRUB installation succeeds. I can > then retry deployment and deployment works fine. I suspect the issue has > to do with a newer version of GRUB being installed. Previous versions of > GRUB must of been similar enough that the PReP partition didn't have to > be fully wiped before installing. That makes more sense; if the layout changed, but I suspect that PReP format isn't really changing. Maybe it's attempting to copy in a new payload and checking. The check for clean prep partition is from 2013, so no new grub code AFAICT. > > @Ryan, as you mentioned Curtin has a bug where it is not fully wiping > the PReP partition which appears to be the root cause of this bug. It > may be worth modifying Curtin to always wipe any partition with the > 'prep' flag set regardless of what wipe is set to to ensure this doesn't > happen in other environments. That seems reasonable, I was adding a wipe after creating if the mode wasn't superblock (as we already clear superblocks before creating partitions to not trigger buried data like raid or bcache). But just ensuring prep partition is wiped works as well. I'll have a branch up soon. I would like to understand what's different on the CI side as the curtin code *hasnt* changed despite there being a bug for not clearing it. > > > [1] > https://git.launchpad.net/maas/tree/src/maasserver/models/partitiontable.py#n152 > [2] https://git.launchpad.net/maas/tree/src/maasserver/preseed_storage.py#n254 > > ** Changed in: curtin >Status: Incomplete => Confirmed > > ** Changed in: grub2 (Ubuntu) >Status: New => Invalid > > ** Changed in: maas >Status: Incomplete => Invalid > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1800153 > > Title: > [2.5] Failed to deploy ppc64el when partition table is GPT > > To manage notifications about this bug go to: > https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
Every time I commission the PPC host we have in our CI I always get a GPT partitioning table using MAAS 2.4.2+. Looking through the source code it appears GPT is always set when creating a new partitioning table[1] and when generating the preseed[2] due to the bios_boot_method being powernv. I'm not sure how msdos got set but I believe the partition table type is a red herring. The issue appears to be that Curtin is not wiping the PReP partition(/dev/sda1). If I chroot into the deployment environment and try to install GRUB with grub-install --target=powerpc-ieee1275 /dev/sda1 --no-nvram it fails. If I wipe /dev/sda1 first GRUB installation succeeds. I can then retry deployment and deployment works fine. I suspect the issue has to do with a newer version of GRUB being installed. Previous versions of GRUB must of been similar enough that the PReP partition didn't have to be fully wiped before installing. @Ryan, as you mentioned Curtin has a bug where it is not fully wiping the PReP partition which appears to be the root cause of this bug. It may be worth modifying Curtin to always wipe any partition with the 'prep' flag set regardless of what wipe is set to to ensure this doesn't happen in other environments. [1] https://git.launchpad.net/maas/tree/src/maasserver/models/partitiontable.py#n152 [2] https://git.launchpad.net/maas/tree/src/maasserver/preseed_storage.py#n254 ** Changed in: curtin Status: Incomplete => Confirmed ** Changed in: grub2 (Ubuntu) Status: New => Invalid ** Changed in: maas Status: Incomplete => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
@Andres can you attatch the full config when you run this under beta3 with the MBR config? >From what I can tell, the prep partition (/dev/sda1) is created fine, but our wipe clears out the first and last 1M , it's an 8M partition. This is a curtin bug as you've specified 'wipe: zero' which should have written zeros over the entire partition. That said, the same thing happens for MBR as well, so unless parted itself is doing "extra" wiping for us; I'm not sure why the MBR version works but the GPT one doesn't. We'll get a patch to ensure the partition handler checks the value of the wipe mode and doesn't assume. On Tue, Oct 30, 2018 at 2:16 PM Andres Rodriguez wrote: > > @Ryan, with curtin verbose: https://pastebin.ubuntu.com/p/nzvHvStyKb/ > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1800153 > > Title: > [2.5] Failed to deploy ppc64el when partition table is GPT > > To manage notifications about this bug go to: > https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
@Ryan, with curtin verbose: https://pastebin.ubuntu.com/p/nzvHvStyKb/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
Can you do a curtin verbose install with GPT and capture the logs from that? I don't think mbr should matter, but we do use 'parted' for msdos partition creation and 'sgdisk' for gpt so there could be some issue going on here. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
>From the MAAS perspective, I can confirm that MAAS is setting to GPT, and it is failing. When setting it to MBR, it succeeds. That said... the other question is whether curtin needs to support using GPT instead? ** Changed in: maas Status: Triaged => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
The difference in behavior is between 2.5.0-beta3 and 418bd67af816b4a99747eb7a1ca8cb966b58ed84 The only storage related change is a UI change: https://git.launchpad.net/maas/commit/?id=32e3d12485456e07fbf5deb82ba6ab455eeefcb5 , which wouldn't seem related. ** Changed in: maas Assignee: (unassigned) => Lee Trager (ltrager) ** Changed in: maas Importance: Undecided => Critical ** Changed in: maas Status: Incomplete => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el when partition table is GPT
After further investigation, it seems that commissioning is setting different partition tables: MAAS 2.5.0beta3 sets partition table to MBR MAAS 2.5.0beta4 sets partition table to GPT -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el
I deployed using MAAS 2.5.0beta3, and the result is successful. This is the config from get-curtin-config: http://paste.ubuntu.com/p/bg4JZ4bVb7/ It seems that 2.5.0beta3 has: - id: sda model: IPR-0 6DC90500 name: sda ptable: msdos serial: IBM_IPR-0_6DC90580 type: disk MAAS from master automatically sets the partition table to: - id: sda model: IPR-0 6DC90500 name: sda ptable: gpt serial: IBM_IPR-0_6DC90580 type: disk wipe: superblock It seems to me that the fact that latest MAAS is changing that to gpt is what's breaking the deployments. ** Summary changed: - [2.5] Failed to deploy ppc64el + [2.5] Failed to deploy ppc64el when partition table is GPT -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el when partition table is GPT To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el
The failed installation log for the above config: https://pastebin.ubuntu.com/p/FSMCXRMzXt/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el
** Attachment added: "storage-ppc64.png" https://bugs.launchpad.net/maas/+bug/1800153/+attachment/5206915/+files/storage-ppc64.png -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el
The curtin configuration can be found here: http://paste.ubuntu.com/p/89DwnJGzc2/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el
** Changed in: curtin Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el
Can you attach the curtin logs? And do you have logs from the last previous successful run? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1800153] Re: [2.5] Failed to deploy ppc64el
** Also affects: grub2 (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1800153 Title: [2.5] Failed to deploy ppc64el To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1800153/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
