Re: [OE-core] The design document for ccache-native
On 6/10/2011 8:09 AM, Mike Westerhof wrote:
> On 6/10/2011 12:48 AM, Robert Yang wrote:
>>
>> On 06/10/2011 11:26 AM, wenzong fan wrote:
>>>
>>>
>>> Original Message
>>>
>>> On Thu, 2011-06-09 at 15:40 -0700, Saul Wold wrote:
>>>> On 06/02/2011 08:11 PM, wenzong fan wrote:
>>>>> Hi Folks,
>>>>>
>>>>> Please help me to review the design document for ccache-native, and
>>>>> I also have two questions about it, any answers or suggestions are
>>>>> appreciated.
>>>>>
>>>>> * Feature name: ccache-native
>>>>> Priority: P3; M2
>>>>> Owner: Wenzong Fan
>>>>> Summary: Integrate ccache-native to yocto
>>>>>
>>>>> * Description:
>>>>> Bitbake supports the 'CCACHE Mechanism', but 'ccache' hasn't been
>>>>> included by poky/yocto, just add it as a native tool.
>>>>>
>>>>> * Usage:
>>>>> Build ccache as a native tool by default and enable it for speeding
>>>>> target packages build.
>>>>>
>>>>> * Implementation:
>>>>> 1) Copy bb file from OE upstream to:
>>>>> meta/recipes-devtools/ccache/
>>>>>
>>>>> 2) Update bb file to get the latest ccache_3.1.5 and split the single
>>>>> bb file to:
>>>>> 'ccache_3.1.5.bb', 'ccache.inc'
>>>>>
>>>>> 3) Enable ccache in the native tools building.
>>>>>
>>>> You will need to have it be a dependency pretty early on in the build.
>>>> Additionally, this is a bit a new part to this task, we want to have the
>>>> default CCACHE_DIR for the build default to a directory in TMPDIR
>>>> instead of the user's home directory. This will mean setting an
>>>> environment variable somewhere early also.
>>>
>>> There is a little more detail on:
>>>
>>> https://wiki.yoctoproject.org/wiki/Yocto_1.1_Schedule
>>>
>>> Specifically, "c) Set CCACHE on a per recipe basis. need to figure out
>>> whether ccache data can be shared and under what circumstances."
>>>
>>> so something like adding:
>>>
>>> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/${PN}"
>>>
>>
>> I think that set CCACHE_DIR on per recipe basis would degrade hit
>> efficiency,
>> the following ccache data can be shared if they they use the same
>> CCACHE_DIR,
>> but if we set CACHE_DIR on a per recipe basis, then they can't be shared:
>>
>> 1) Most pkg's configure will run "gcc foo.c" for checking the C compiler,
>>these compiling are the same between different pkgs at most time.
>>
>> 2) Some recipes' compiling are similar, for example: gcc-cross,
>>gcc-corss-initial and gcc-cross-intermediate.
>>
>> I think that:
>>
>> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/"
>>
>> would be better.
>
> I fear that the use of ccache is inherently risky with OE. Given the
> (relatively common) case where the user blows away their TMPDIR in order
(Replying to my own message, because I didn't have enough coffee to be
clear in the original!) I'm agreeing with the suggestion that it be in
TMPDIR, and offering the suggestion that if we don't put it there, we
need to qualify it with the toolchain version. (Now I'm going to get
another cup of coffee -- sorry for any confusion!)
-Mike (mwester)
> to get a full, clean rebuild after an update to the toolchain is make,
> it is possible that ccache will erroneously re-use an object created by
> the earlier version of the toolchain.
>
> While imperfect, I would suggest that we would do better if we would
> somehow embed the PV (or something like that) for the toolchain into the
> CCACHE_DIR, thus ensuring that we don't risk the re-use of old objects
> when the toolchain is updated.
>
> (Frankly, given my experiences with it, I'd prefer we just disable
> ccache entirely with OE.)
>
> -Mike (mwester)
>
>> // Robert
>>
>>> to bitbake.conf with a bit more thought into working out the right
>>> components to add to the variable.
>>>
>>> Cheers,
>>>
>>> Richard
>>>
>>>
>>>
>>>
>>
>> ___
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>>
>>
>
>
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] The design document for ccache-native
On 6/10/2011 12:48 AM, Robert Yang wrote:
>
> On 06/10/2011 11:26 AM, wenzong fan wrote:
>>
>>
>> Original Message
>>
>> On Thu, 2011-06-09 at 15:40 -0700, Saul Wold wrote:
>>> On 06/02/2011 08:11 PM, wenzong fan wrote:
>>> > Hi Folks,
>>> >
>>> > Please help me to review the design document for ccache-native, and
>>> > I also have two questions about it, any answers or suggestions are
>>> > appreciated.
>>> >
>>> > * Feature name: ccache-native
>>> > Priority: P3; M2
>>> > Owner: Wenzong Fan
>>> > Summary: Integrate ccache-native to yocto
>>> >
>>> > * Description:
>>> > Bitbake supports the 'CCACHE Mechanism', but 'ccache' hasn't been
>>> > included by poky/yocto, just add it as a native tool.
>>> >
>>> > * Usage:
>>> > Build ccache as a native tool by default and enable it for speeding
>>> > target packages build.
>>> >
>>> > * Implementation:
>>> > 1) Copy bb file from OE upstream to:
>>> > meta/recipes-devtools/ccache/
>>> >
>>> > 2) Update bb file to get the latest ccache_3.1.5 and split the single
>>> > bb file to:
>>> > 'ccache_3.1.5.bb', 'ccache.inc'
>>> >
>>> > 3) Enable ccache in the native tools building.
>>> >
>>> You will need to have it be a dependency pretty early on in the build.
>>> Additionally, this is a bit a new part to this task, we want to have the
>>> default CCACHE_DIR for the build default to a directory in TMPDIR
>>> instead of the user's home directory. This will mean setting an
>>> environment variable somewhere early also.
>>
>> There is a little more detail on:
>>
>> https://wiki.yoctoproject.org/wiki/Yocto_1.1_Schedule
>>
>> Specifically, "c) Set CCACHE on a per recipe basis. need to figure out
>> whether ccache data can be shared and under what circumstances."
>>
>> so something like adding:
>>
>> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/${PN}"
>>
>
> I think that set CCACHE_DIR on per recipe basis would degrade hit
> efficiency,
> the following ccache data can be shared if they they use the same
> CCACHE_DIR,
> but if we set CACHE_DIR on a per recipe basis, then they can't be shared:
>
> 1) Most pkg's configure will run "gcc foo.c" for checking the C compiler,
>these compiling are the same between different pkgs at most time.
>
> 2) Some recipes' compiling are similar, for example: gcc-cross,
>gcc-corss-initial and gcc-cross-intermediate.
>
> I think that:
>
> export CCACHE_DIR = "${TMPDIR}/ccache/${TARGET_SYS}/"
>
> would be better.
I fear that the use of ccache is inherently risky with OE. Given the
(relatively common) case where the user blows away their TMPDIR in order
to get a full, clean rebuild after an update to the toolchain is make,
it is possible that ccache will erroneously re-use an object created by
the earlier version of the toolchain.
While imperfect, I would suggest that we would do better if we would
somehow embed the PV (or something like that) for the toolchain into the
CCACHE_DIR, thus ensuring that we don't risk the re-use of old objects
when the toolchain is updated.
(Frankly, given my experiences with it, I'd prefer we just disable
ccache entirely with OE.)
-Mike (mwester)
> // Robert
>
>> to bitbake.conf with a bit more thought into working out the right
>> components to add to the variable.
>>
>> Cheers,
>>
>> Richard
>>
>>
>>
>>
>
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>
>
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Re: [OE-core] Where to host new layers
On 5/13/2011 4:10 AM, Khem Raj wrote: > Hi Mike and all > > I have created couple of layers namely meta-slugos (distro layer) and > meta-nslu ( machine layer). I was looking for place to host those > layers. Would they be good under meta-openembedded ? I dont think > http://www.nslu2-linux.org/ > has git infrastructure otherwise that would be one place to put it. > > Thanks > > -Khem Thanks Khem! Tom has indicated that he'll set up the appropriate GIT infrastructure to properly host these. I'm still traveling, but should be back next week -- so that would be a good time to see if we can move the repos from the github location they're currently at. Again, thanks for doing this! -Mike (mwester) ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
