On 12/8/25 11:31, Alexander Kanavin wrote: > On Fri, 5 Dec 2025 at 19:12, Gyorgy Sarvari <[email protected]> wrote: >>>> If fetch is deemed successful, it is stored in the shared state cache, >>>> and then it is not repeated until the relevant signature changes - I >>>> think this is the metadata reference you found. >>>> However as long as the download folder's content is only modified by one >>>> project only, it supposed keep track of it fine. >>>> >>>> One question about the DL_DIR: is it possible that it is shared between >>>> multiple projects, while keeping separate shared state cache? Or that >>>> the DL_DIR's content modified manually? >>>> The shared state cache supposed to keep track of what is downloaded by >>>> bitbake and what is not, but if the DL_DIR's content is changed without >>>> bitbake (or by a separate bitbake), that can break the shared state, and >>>> cause such issues. >>> This is not correct. Shared state does not track what is downloaded. >> It depends on how you interpret it. Sstate does track the do_fetch task, >> and does know if it needs to re-executed or not. > The process by which bitbake determines which tasks need to be > executed, and in which order, is well defined, and it is not like > that. > > This is how it is: > 1. Bitbake unrolls the target specified on a command line into a > directed graph, where nodes are tasks and edges are task dependencies. > Each task is given a signature with a hash. > 2. For each task bitbake checks if there's a stamp in the build > directory with a matching hash (meaning the build directory already > has the outcome of the task), and if so, the task, and everything it > depends on, is pruned from the graph. > 3. For each task that is sstate-enabled (neither fetch nor unpack > tasks are in that set), bitbake checks if sstate contains an object > matching the task signature. If so, everything that the task depends > on is pruned from the graph, and the task is replaced with its > setscene variant that unpacks the sstate. > 4. Remaining tasks are executed in order that respects their dependencies. > > Sstate cache (or code that manages it) does not track tasks, it only > stores and retrieves sstate objects, and it does not know anything > about what tasks in a particular build need to be executed. > >>> It is also fine to share DL_DIR between multiple builds and projects. >> I did not write that it is not fine to share DL_DIR, but but you need to >> know some caveats. If you run bitbake -c cleanall in one project, you >> are up for a good time in the other one, if they use the same recipe. > That is fine to do as well, as long as no other builds are running at > the same time as cleanall. The other project either does not need to > run fetch/unpack and won't notice sources are absent, or it will run > fetch, fetch will notice the sources are absent in DL_DIR, and then it > will simply re-fetch them.
1. Set up a new project project1, with a custom dl_dir 2. Set up a new project project2, with the same custom dl_dir 3. Run bitbake -c fetch $RECIPE in both project1 4. Run bitbake -c fetch $RECIPE in both project2 5. Run bitbake -c cleanall $RECIPE in project1 6. Run bitbake $RECIPE in project2. Once again - using shared DL_DIR is fine usually (and I never even implied the opposite of it), but there are gotchas. On a related note: this is not an order of course, but if you could not reply, that would be great. I will also go out of my way to avoid direct communication with you now, and in the future. > Alex
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#66100): https://lists.yoctoproject.org/g/yocto/message/66100 Mute This Topic: https://lists.yoctoproject.org/mt/116612004/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
