Re: [yocto] Working on Dependent Recipes in eSDK
On Thu, Oct 25, 2018 at 8:29 AM wrote: > > > ChenQi wrote on 10/22/2018 07:04:32 PM: > > > > Hi Aaron, > > > > The main pain for now is that `devtool sdk-install ' does > > > not work as expected, right? > > I just checked the codes of sdk-install, and found that it does not fit > > the current RSS design. > > I'd like to check with you if this is the major concern from your team? > > Do you have some other concerns? > > I'll open enhancements/bugs on bugzilla and hopefully will work on them > > in the 2.7 development cycle. > > > > Best Regards, > > Chen Qi > > > > ... snip ... > > Well, I'm not quite sure how "devtool sdk-install " > relates after "devtool modify " has been called. > > My developers use devtool to install dependent recipes into the eSDK when > they start working on something. Often they find that they need to make a > change in the library code that a dependent recipe provides. In effect, > they are developing a patch for the dependent recipe. So they use devtool > to say they're going to modify that dependent recipe, and they go and > make the changes they want. Then what? How do they get those changes, > that patch, available to the eSDK? > > Any changes made to the library code provided by the dependent recipe are > not available to the eSDK, except through devtool commands. In effect > there are two different worlds at play, the eSDK sysroots versus the > recipe sysroots. The devtool commands seem to make use of the recipe > sysroots, but in my mind that defeats the purpose of an eSDK. So I made > a plug-in for devtool were I can force the eSDK sysroots to be updated. > I'm not sure how that works, but it seems to propagate the workspace > (recipe sysroot land) to the eSDK, so changes that developers make are > now available for use. > > I want my developers to use the toolchain (cmake, make, gcc, etc) > provided by the eSDK and by things installed into it > (devtool sdk-install), rather than having to use devtool as their > iterative build command. > > Ultimately, I would like to be able to skip my devtool plug-in, as it > requires the use of "devtool build " and > "devtool sdk-sysroot" each time a dependent recipe is modified. But I am > not sure how that would work, other than the devtool build command > should be updating the eSDK sysroots itself. I think you have two options here 1. Feedback the tweaks to packages back into source and re-generate SDK and if you setup SDK feeds then users/devs can simply do devtool sdk-update and this time the package in question will get updated in their SDKs with the change that was initiated. second option could be to generate a downstream derivative SDK using devtool build-sdk and use this SDK > -- > ___ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Working on Dependent Recipes in eSDK
> ChenQi wrote on 10/22/2018 07:04:32 PM: > > Hi Aaron, > > The main pain for now is that `devtool sdk-install ' does > not work as expected, right? > I just checked the codes of sdk-install, and found that it does not fit > the current RSS design. > I'd like to check with you if this is the major concern from your team? > Do you have some other concerns? > I'll open enhancements/bugs on bugzilla and hopefully will work on them > in the 2.7 development cycle. > > Best Regards, > Chen Qi > > ... snip ... Well, I'm not quite sure how "devtool sdk-install " relates after "devtool modify " has been called. My developers use devtool to install dependent recipes into the eSDK when they start working on something. Often they find that they need to make a change in the library code that a dependent recipe provides. In effect, they are developing a patch for the dependent recipe. So they use devtool to say they're going to modify that dependent recipe, and they go and make the changes they want. Then what? How do they get those changes, that patch, available to the eSDK? Any changes made to the library code provided by the dependent recipe are not available to the eSDK, except through devtool commands. In effect there are two different worlds at play, the eSDK sysroots versus the recipe sysroots. The devtool commands seem to make use of the recipe sysroots, but in my mind that defeats the purpose of an eSDK. So I made a plug-in for devtool were I can force the eSDK sysroots to be updated. I'm not sure how that works, but it seems to propagate the workspace (recipe sysroot land) to the eSDK, so changes that developers make are now available for use. I want my developers to use the toolchain (cmake, make, gcc, etc) provided by the eSDK and by things installed into it (devtool sdk-install), rather than having to use devtool as their iterative build command. Ultimately, I would like to be able to skip my devtool plug-in, as it requires the use of "devtool build " and "devtool sdk-sysroot" each time a dependent recipe is modified. But I am not sure how that would work, other than the devtool build command should be updating the eSDK sysroots itself. -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Working on Dependent Recipes in eSDK
Hi Aaron, The main pain for now is that `devtool sdk-install ' does not work as expected, right? I just checked the codes of sdk-install, and found that it does not fit the current RSS design. I'd like to check with you if this is the major concern from your team? Do you have some other concerns? I'll open enhancements/bugs on bugzilla and hopefully will work on them in the 2.7 development cycle. Best Regards, Chen Qi On 10/19/2018 04:59 AM, aaron_wri...@selinc.com wrote: Paul Eggleton wrote on 10/18/2018 12:30:07 PM: I think we would *like* this to be possible, from what I can tell I think the issue is that the copies of the library in the sysroots of other dependent recipes aren't being updated. The eSDK (and devtool) were originally written when there weren't recipe-specific sysroots, and this stuff was a bit simpler. Moving back to the full build system won't automatically fix this, though it does make workarounds possible - you'd need to rebuild the dependent recipes to get their sysroots updated. Cheers, Paul ... snip ... I put together a devtool plugin to almost do what I want: from devtool import exec_build_env_command, DevtoolError def sdk_sysroot(args, config, basepath, workspace): """Entry point for the devtool sdk-sysroot command""" try: exec_build_env_command(config.init_path, basepath, 'bitbake build-sysroots', watch=True) except bb.process.ExecutionError as e: raise DevtoolError('Failed to bitbake build-sysroots:\n%s' % (str(e))) def register_commands(subparsers, context): """Register devtool subcommands from the sdk plugin""" if context.fixed_setup: parser_sdk_sysroot = subparsers.add_parser( 'sdk-sysroot', help='Update the SDK sysroot', description='Updates the SDK sysroot so that changes made to recipes in the workspace will be available,', group='sdk') parser_sdk_sysroot.set_defaults(func=sdk_sysroot) This allows me to "devtool modify ", edit the code, "devtool build ", "devtool sdk-sysroot", and use the new code from the dependent recipe immediately. This makes me wonder if "bitbake build-sysroots" should not be a part of "devtool build", like it is for "devtool sdk-install" (where I stole it from). -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Working on Dependent Recipes in eSDK
> Paul Eggleton wrote on 10/18/2018 12:30:07 PM: > > I think we would *like* this to be possible, from what I can tell I think the > issue is that the copies of the library in the sysroots of other dependent > recipes aren't being updated. The eSDK (and devtool) were originally written > when there weren't recipe-specific sysroots, and this stuff was a bit simpler. > > Moving back to the full build system won't automatically fix this, though it > does make workarounds possible - you'd need to rebuild the dependent recipes > to get their sysroots updated. > > Cheers, > Paul > ... snip ... I put together a devtool plugin to almost do what I want: > from devtool import exec_build_env_command, DevtoolError > > def sdk_sysroot(args, config, basepath, workspace): > """Entry point for the devtool sdk-sysroot command""" > > try: > exec_build_env_command(config.init_path, basepath, 'bitbake build-sysroots', watch=True) > except bb.process.ExecutionError as e: > raise DevtoolError('Failed to bitbake build-sysroots:\n%s' % (str(e))) > > def register_commands(subparsers, context): > """Register devtool subcommands from the sdk plugin""" > if context.fixed_setup: > parser_sdk_sysroot = subparsers.add_parser( > 'sdk-sysroot', > help='Update the SDK sysroot', > description='Updates the SDK sysroot so that changes made to recipes in the workspace will be available,', > group='sdk') > parser_sdk_sysroot.set_defaults(func=sdk_sysroot) This allows me to "devtool modify ", edit the code, "devtool build ", "devtool sdk-sysroot", and use the new code from the dependent recipe immediately. This makes me wonder if "bitbake build-sysroots" should not be a part of "devtool build", like it is for "devtool sdk-install" (where I stole it from). -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Working on Dependent Recipes in eSDK
> Alexander Kanavin wrote on 10/18/2018 11:30:17 AM: > > I don’t think esdk is designed for this kind of ‘full stack’ > development. You can go back to working directly with yocto layers, > and skip the esdk altogether. Devtool will work exactly same (minus > the sdk specific commands). > > Alex I'm trying to get the developers away from using bitbake and devtool to build their code, as those tools have overhead and don't easily allow the developers to iterate quickly or allow them to run other make targets for unit testing, etc. I really want the developers at the "make" level for the majority of their time. When it comes time for recipes and image creation, then they'll use devtool. > > On 18 Oct 2018, at 19.37, aaron_wri...@selinc.com wrote: > > > > My team recently started using the eSDK for day-to-day development. > > Previously, they were using the bitbake command to build, which works, but > > has overhead. Yesterday three different people came up to me a question > > about workflow with the eSDK, and they all had to do with dependent > > recipes in the eSDK. > > > > For example, a developer would sit down to make something new that relies > > on a library provided by a recipe in the eSDK. So they "devtool > > sdk-install needed-library", and get to work. While working, they > > discover, I need to add something to this library I depend on so that it > > does what I want in the way I want. So they "devtool modify > > needed-library" and change the source of that library to do what they > > want, and build that library using its build system (CMake or Autotools). > > Then they return to working on the new something and find that none of > > their changes to the library they depend on are available. > > > > At this point, they are lost. They just start trying things to make it > > work, like "devtool build needed-library", which successfully completes, > > but doesn't help. Some people also try to use the build system of the > > library they modified to install it into the machine sysroot of the eSDK > > with a command like, "make install > > DESTDIR=/tmp/sysroots/", and that actually does help. > > They can now use the changes they made to the library in the new something > > they are working on. However, when they use the "devtool sdk-install" > > command again, their changes to the needed library that they installed > > into the machine sysroot are reverted. > > > > So--what is the workflow they are supposed to use? > > > > Pushing the changes to the library upstream, updating the recipe, and > > waiting for a new eSDK is not practical given that the developers may have > > to iterate on the changes to the library quite a bit before they get it > > just right for their new something. > > > > Is there a way to update the eSDK sysroots with recipes that are in the > > workspace? Could "devtool build" also update the eSDK sysroots? Or is > > there a different way to get this done? > > -- > > ___ > > yocto mailing list > > yocto@yoctoproject.org > > https://urldefense.proofpoint.com/v2/url? > u=https-3A__lists.yoctoproject.org_listinfo_yocto&d=DwIFaQ&c=- > _uRSsrpJskZgEkGwdW- > sXvhn_FXVaEGsm0EI46qilk&r=GBJubmJUwwrr1B4TJFRqESMrRsDipvY66vPwm4- > gtNQ&m=- > C-70rjnGksaogzIkgE5MdoY3IfA3x62MZVOwbZmw4c&s=T5PxZ3bNB429M76sWwBqgrtMsMhB- > kz1BXkdY8hOIRY&e= -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Working on Dependent Recipes in eSDK
I think we would *like* this to be possible, from what I can tell I think the issue is that the copies of the library in the sysroots of other dependent recipes aren't being updated. The eSDK (and devtool) were originally written when there weren't recipe-specific sysroots, and this stuff was a bit simpler. Moving back to the full build system won't automatically fix this, though it does make workarounds possible - you'd need to rebuild the dependent recipes to get their sysroots updated. Cheers, Paul On Friday, 19 October 2018 7:30:17 AM NZDT Alexander Kanavin wrote: > I don’t think esdk is designed for this kind of ‘full stack’ development. > You can go back to working directly with yocto layers, and skip the esdk > altogether. Devtool will work exactly same (minus the sdk specific > commands). > > Alex > > > On 18 Oct 2018, at 19.37, aaron_wri...@selinc.com wrote: > > > > My team recently started using the eSDK for day-to-day development. > > Previously, they were using the bitbake command to build, which works, but > > has overhead. Yesterday three different people came up to me a question > > about workflow with the eSDK, and they all had to do with dependent > > recipes in the eSDK. > > > > For example, a developer would sit down to make something new that relies > > on a library provided by a recipe in the eSDK. So they "devtool > > sdk-install needed-library", and get to work. While working, they > > discover, I need to add something to this library I depend on so that it > > does what I want in the way I want. So they "devtool modify > > needed-library" and change the source of that library to do what they > > want, and build that library using its build system (CMake or Autotools). > > Then they return to working on the new something and find that none of > > their changes to the library they depend on are available. > > > > At this point, they are lost. They just start trying things to make it > > work, like "devtool build needed-library", which successfully completes, > > but doesn't help. Some people also try to use the build system of the > > library they modified to install it into the machine sysroot of the eSDK > > with a command like, "make install > > DESTDIR=/tmp/sysroots/", and that actually does help. > > They can now use the changes they made to the library in the new something > > they are working on. However, when they use the "devtool sdk-install" > > command again, their changes to the needed library that they installed > > into the machine sysroot are reverted. > > > > So--what is the workflow they are supposed to use? > > > > Pushing the changes to the library upstream, updating the recipe, and > > waiting for a new eSDK is not practical given that the developers may have > > to iterate on the changes to the library quite a bit before they get it > > just right for their new something. > > > > Is there a way to update the eSDK sysroots with recipes that are in the > > workspace? Could "devtool build" also update the eSDK sysroots? Or is > > there a different way to get this done? > -- Paul Eggleton Intel Open Source Technology Centre -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Working on Dependent Recipes in eSDK
I don’t think esdk is designed for this kind of ‘full stack’ development. You can go back to working directly with yocto layers, and skip the esdk altogether. Devtool will work exactly same (minus the sdk specific commands). Alex > On 18 Oct 2018, at 19.37, aaron_wri...@selinc.com wrote: > > My team recently started using the eSDK for day-to-day development. > Previously, they were using the bitbake command to build, which works, but > has overhead. Yesterday three different people came up to me a question > about workflow with the eSDK, and they all had to do with dependent > recipes in the eSDK. > > For example, a developer would sit down to make something new that relies > on a library provided by a recipe in the eSDK. So they "devtool > sdk-install needed-library", and get to work. While working, they > discover, I need to add something to this library I depend on so that it > does what I want in the way I want. So they "devtool modify > needed-library" and change the source of that library to do what they > want, and build that library using its build system (CMake or Autotools). > Then they return to working on the new something and find that none of > their changes to the library they depend on are available. > > At this point, they are lost. They just start trying things to make it > work, like "devtool build needed-library", which successfully completes, > but doesn't help. Some people also try to use the build system of the > library they modified to install it into the machine sysroot of the eSDK > with a command like, "make install > DESTDIR=/tmp/sysroots/", and that actually does help. > They can now use the changes they made to the library in the new something > they are working on. However, when they use the "devtool sdk-install" > command again, their changes to the needed library that they installed > into the machine sysroot are reverted. > > So--what is the workflow they are supposed to use? > > Pushing the changes to the library upstream, updating the recipe, and > waiting for a new eSDK is not practical given that the developers may have > to iterate on the changes to the library quite a bit before they get it > just right for their new something. > > Is there a way to update the eSDK sysroots with recipes that are in the > workspace? Could "devtool build" also update the eSDK sysroots? Or is > there a different way to get this done? > -- > ___ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] Working on Dependent Recipes in eSDK
My team recently started using the eSDK for day-to-day development. Previously, they were using the bitbake command to build, which works, but has overhead. Yesterday three different people came up to me a question about workflow with the eSDK, and they all had to do with dependent recipes in the eSDK. For example, a developer would sit down to make something new that relies on a library provided by a recipe in the eSDK. So they "devtool sdk-install needed-library", and get to work. While working, they discover, I need to add something to this library I depend on so that it does what I want in the way I want. So they "devtool modify needed-library" and change the source of that library to do what they want, and build that library using its build system (CMake or Autotools). Then they return to working on the new something and find that none of their changes to the library they depend on are available. At this point, they are lost. They just start trying things to make it work, like "devtool build needed-library", which successfully completes, but doesn't help. Some people also try to use the build system of the library they modified to install it into the machine sysroot of the eSDK with a command like, "make install DESTDIR=/tmp/sysroots/", and that actually does help. They can now use the changes they made to the library in the new something they are working on. However, when they use the "devtool sdk-install" command again, their changes to the needed library that they installed into the machine sysroot are reverted. So--what is the workflow they are supposed to use? Pushing the changes to the library upstream, updating the recipe, and waiting for a new eSDK is not practical given that the developers may have to iterate on the changes to the library quite a bit before they get it just right for their new something. Is there a way to update the eSDK sysroots with recipes that are in the workspace? Could "devtool build" also update the eSDK sysroots? Or is there a different way to get this done? -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto