ssssam opened a new issue, #2081: URL: https://github.com/apache/buildstream/issues/2081
BuildStream is infinitely extensible using plugins written in Python. That's really great, but problems in plugins lead to broken builds, and it can give the impression that BuildStream itself is unreliable. As an example, consider the log output from https://github.com/endlessm/eos-build-meta/issues/163 (edited for brevity): ``` gnome-build-meta.bst:gnomeos-deps/zram-generator.bst: [00:00:01][][ fetch:gnome-build-meta.bst:gnomeos-deps/zram-generator.bst] BUG Fetch An unhandled exception occured: Traceback (most recent call last): File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/jobs/job.py", line 350, in child_action result = self.child_process() # pylint: disable=assignment-from-no-return ^^^^^^^^^^^^^^^^^^^^ File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/jobs/elementjob.py", line 81, in child_process return self._action_cb(self._element) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/_scheduler/queues/fetchqueue.py", line 75, in _fetch_not_original element._fetch(fetch_original=False) ... File "/home/runner/_work/eos-build-meta/eos-build-meta/venv/lib/python3.12/site-packages/buildstream/source.py", line 1508, in _stage self.stage(directory) File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/git_repo.py", line 125, in stage mirror.stage(directory) File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/_git_utils.py", line 313, in stage objects = collect_objects(mirror, commit, base, True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/_work/eos-build-meta/eos-build-meta/.bst/staged-junctions/gnome-build-meta.bst/b200c68d253197773709f06ac0f852456110fb0eb7ebd6a7d2ce2a76e24a18a1/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/371987227b68e9af8b7315c41cab2a552393442f9d2ffa5cdabded47f3e30667/src/buildstream_plugins_community/sources/_git_utils.py", line 73, in collect_objects entry.commit for entry in mirror.get_walker(commit.id, excluded) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: BaseRepo.get_walker() takes from 1 to 2 positional arguments but 3 were given ``` To the casual reader, this log says: *there is a BUG, somewhere in BuildStream's Python code*. With training in how to read the log, you might figure out the nuance: *there is a bug somewhere in [buildstream-plugins-community](https://gitlab.com/BuildStream/buildstream-plugins-community)*. And if you are one of the elite group of people who have worked on buildstream-plugins-community before, you might see this and realize: *this is a problem caused by the [dulwich](https://dulwich.io/) python library*. If you are in the 99.9% of people who don't know much about BuildStream's internals, it simply looks BuildStream is badly coded and it crashes at random on some machines --- which is really unfortunate because BuildStream aims to achieve quite the opposite, and make builds *repeatable* on different machines. Since anyone can write a plugin, there is no way to avoid plugins failing some of the time. What is missing is a way to clearly identify *which plugin caused the failure*. Implementing this probably requires some extra code around every point that BuildStream calls into a plugin, which should be non-trivial but doable. I think it's better for everyone if the log output in this kind of case looks more like: ``` [00:00:01][][ fetch:gnome-build-meta.bst:gnomeos-deps/zram-generator.bst] PLUGIN ERROR Fetch (using `git_repo` source plugin) An unhandled exception occured in the `git_repo` source plugin: Traceback (most recent call last): .... TypeError: BaseRepo.get_walker() takes from 1 to 2 positional arguments but 3 were given This is not a problem with BuildStream's core -- report it to the developers of the plugin. The `git_repo` source plugin was provided by junction element: `freedesktop-sdk.bst:plugins/buildstream-plugins.bst`. ``` The intention is to make it clear that BuildStream's core is working as expected, and the user needs to consider their choice of plugins. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
