Hi Paul
> Hi Paul,
>
> Thanks for identifying the root cause.
>
> After studying the devtool design, I now understand what happens: when
> devtool modify creates the workspace, it does not copy the full git object
> store
> locally.
> Instead, it clones the source tree using git's alternate object store
> mechanism -
> the workspace's .git/objects/info/alternates file points back to the
> downloaded
> bare repositories under the downloads/ directory.
> This way the workspace shares the git objects with the download cache rather
> than duplicating them on disk.
>
> When bitbake -c cleanall is run, it deletes the downloaded bare repositories
> from downloads/. The workspace source tree still exists, but its alternates
> now
> point to missing paths.
> Any git operation on the workspace (including git status, git add, etc.) then
> fails because git cannot resolve the object references, producing errors like:
>
> error: unable to normalize alternate object path:
> .../downloads/git2/.../objects
> fatal: bad object HEAD
>
> This is what causes the parser to halt with exit 128.
>
> The fix on my side is straightforward: replace bitbake -c cleanall with
> bitbake
> -c cleansstate in my devtool workflow. cleansstate clears the sstate cache
> without touching the downloads directory, so the workspace alternates remain
> intact.
>
> That said, I am not sure whether some improvement on the OE side would still
> be worthwhile. Currently, when this breakage occurs, the user sees only a
> cryptic CalledProcessError: exit 128 with no indication of what went wrong or
> how to recover.
> The situation is also not obvious: running bitbake -c cleanall on a
> single-repo
> recipe with a devtool workspace does not trigger this problem, so users
> working with multi-repo recipes (such as Zephyr-based recipes that fetch
> multiple git repositories into separate destsuffix directories) are likely to
> hit
> this unexpectedly. A clearer error message pointing to the root cause and the
> recovery steps (devtool reset -n followed by devtool modify) might save other
> users from the same confusion.
>
> Please drop this patch series because it is not a normal test case for
> devtool.
>
> Below patch that may help improve the user experience, though I leave it to
> the community to decide whether they are worth reviewing:
>
> Best regards,
> Jamin
>
> diff --git a/meta/classes/externalsrc.bbclass
> b/meta/classes/externalsrc.bbclass
> index 902ff2604f..3e6c9d937e 100644
> --- a/meta/classes/externalsrc.bbclass
> +++ b/meta/classes/externalsrc.bbclass
> @@ -227,6 +227,57 @@ def srctree_hash_files(d, srcdir=None):
>
> ret = " "
> if git_dir is not None:
> + # Find nested git repos created by multiple SRC_URI git entries with
> + # different destsuffix values, so their alternates can also be
> checked
> + # for breakage below.
> + nested_git_dirs = []
> + for root, dirs, files in os.walk(s_dir):
> + if root == s_dir:
> + continue
> + if '.git' in dirs or '.git' in files:
> + nested_git_dirs.append(root)
> + dirs[:] = []
> +
> + # Check for broken git alternates in the top-level repo and all
> nested
> + # repos. This can happen when 'bitbake -c cleanall' deletes the
> + # downloads directory that the devtool workspace references via
> + # alternates, causing all subsequent git operations to exit 128 and
> + # halt the parse phase. Relative alternate paths are resolved
> against
> + # the repo's objects directory; double-slash paths (a known devtool
> + # artefact) are normalised before the existence check.
> + def has_broken_alternates(dot_git):
> + obj_dir = os.path.join(dot_git, 'objects')
> + alt_file = os.path.join(obj_dir, 'info', 'alternates')
> + if not os.path.exists(alt_file):
> + return False
> + with open(alt_file) as f:
> + for line in f:
> + path = line.strip()
> + if not path or path.startswith('#'):
> + continue
> + if not os.path.isabs(path):
> + path = os.path.join(obj_dir, path)
> + if not os.path.exists(os.path.normpath(path)):
> + return True
> + return False
> +
> + broken = has_broken_alternates(git_dir)
> + if not broken:
> + for nested in nested_git_dirs:
> + nested_dot_git = os.path.join(nested, '.git')
> + if os.path.isdir(nested_dot_git) and
> has_broken_alternates(nested_dot_git):
> + broken = True
> + break
> +
> + if broken:
> + bb.warn('%s: devtool workspace has broken git alternates, '
> + 'likely caused by "bitbake -c cleanall" removing the '
> + 'downloads directory. '
> + 'To repair the workspace, run: '
> + 'devtool reset %s && devtool modify %s'
> + % (d.getVar('PN'), d.getVar('PN'), d.getVar('PN')))
> + return s_dir + '/*:True'
> +
>
Please ignore the comments above.
I believe I have identified the root cause. Please see my updated solution here:
https://patchwork.yoctoproject.org/project/oe-core/patch/[email protected]/
Thanks,
Jamin
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#237383):
https://lists.openembedded.org/g/openembedded-core/message/237383
Mute This Topic: https://lists.openembedded.org/mt/119327122/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
