Sent from mobile phone
> 在 2018年11月22日,20:15,Richard Purdie <richard.pur...@linuxfoundation.org> 写道:
>
>> On Thu, 2018-11-22 at 19:51 +0800, Robert Yang wrote:
>> Leave the task in SSTATETASKS doesn't make sense when it is removed,
>> so also
>> remove it from SSTATETASKS.
>>
>> This can improve the performance a lot for "bitbake <recipe-
>> native/cross/crosssdk>
>> -ccleansstate" when there are a lot of sstate files.
>>
>> For example:
>> * Before
>> $ bitbake quilt-native -ccleansstate
>> - Check log.do_cleansstate:
>> Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_package.tgz*
>> Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_package_qa.tgz*
>> Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_package_write_rpm.tgz*
>> Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_packagedata.tgz*
>> Removing /sstate-cache/*/sstate:quilt-
>> native::0.65:r0::3:*_populate_lic.tgz*
>> Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*
>>
>> There are no package tasks for quilt-native, so the first 4 lines
>> doesn't
>> make any sense, but the glob pattern "sstate-cache/*/*" is very
>> time
>> consuming when there are no disk caches. E.g., I have more than
>> 600,000
>> sstate files:
>> - Without disk caches
>> # echo 3 >/proc/sys/vm/drop_caches
>> $ time python3 -c 'import glob; glob.glob("/sstate-
>> cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_package.tgz*")'
>> real 4m32.583s
>> user 0m5.768s
>> sys 0m12.892s
>>
>> - With disk caches (e.g., run it in the second time)
>> $ time python3 -c 'import glob; glob.glob("/sstate-
>> cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_package.tgz*")'
>> real 0m5.128s
>> user 0m2.772s
>> sys 0m2.308s
>>
>> So the 4 removing *package* commands cost more than 20s or 272s in
>> theory.
>>
>> * After
>> $ bitbake quilt-native -ccleansstate
>> - Check log.do_cleansstate:
>> Removing /sstate-cache/*/sstate:quilt-
>> native::0.65:r0::3:*_populate_lic.tgz*
>> Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-
>> linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*
>>
>> Now the package tasks are gone when cleanssate for native recipes,
>> which
>> saved 20s or 272s in theory.
>>
>> Signed-off-by: Robert Yang <liezhi.y...@windriver.com>
>> ---
>> meta/classes/nopackages.bbclass | 35 +++++++++++++++++++++++------
>> ------
>> 1 file changed, 23 insertions(+), 12 deletions(-)
>>
>> diff --git a/meta/classes/nopackages.bbclass
>> b/meta/classes/nopackages.bbclass
>> index 559f507..66e4936 100644
>> --- a/meta/classes/nopackages.bbclass
>> +++ b/meta/classes/nopackages.bbclass
>> @@ -1,12 +1,23 @@
>> -deltask do_package
>> -deltask do_package_write_rpm
>> -deltask do_package_write_ipk
>> -deltask do_package_write_deb
>> -deltask do_package_qa
>> -deltask do_packagedata
>> -deltask do_package_setscene
>> -deltask do_package_write_rpm_setscene
>> -deltask do_package_write_ipk_setscene
>> -deltask do_package_write_deb_setscene
>> -deltask do_package_qa_setscene
>> -deltask do_packagedata_setscene
>> +NO_PACKAGES_TASKS = " \
>> + do_package \
>> + do_package_write_rpm \
>> + do_package_write_ipk \
>> + do_package_write_deb \
>> + do_package_qa \
>> + do_packagedata \
>> +"
>> +
>> +python() {
>> + sstatetasks = (d.getVar('SSTATETASKS') or '').split()
>> + for task in d.getVar('NO_PACKAGES_TASKS').split():
>> + setscene_task = '%s_setscene' % task
>> + tasks = (task, setscene_task)
>> + for t in tasks:
>> + bb.build.deltask(t, d)
>> +
>> + if task in sstatetasks:
>> + bb.debug(2, 'Removing sstate task %s from SSTATETASKS' %
>> task)
>> + sstatetasks.remove(task)
>> + d.setVar('SSTATETASKS', ' '.join(sstatetasks))
>> +
>> +}
>
> I really don't want to turn more comparatively readable classes into a
> mess of anonymous python if we can help it.
>
> Can we not just teach cleansstate to check if a task exists before
> trying to delete the sstate? It should be possible with
> d.getVarFlag(taskname, "task")?
Thanks, yes, this is my first thought, but I only knew __BBTASKS which is an
internal var of bitbake, I will use d.getVarFlag(taskname, "task") to send a v2.
// Robert
>
> Cheers,
>
> Richard
>
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core