[yocto] Migration 1.8 -> 2.0 opkg-cl missing in SDK

[mar.krzeminski]

Hello,

After migrating from 1.8 to 2.0 opkg-cl is missing in SDK (sdk - is 
generated by bitbake meta-toolchain command).

Manual clearly states that opkg-cl is still avaliable:
http://www.yoctoproject.org/docs/latest/adt-manual/adt-manual.html#configuring-the-pms
I can not find any instruction how to migrate PMS based on opkg in sdk 
in 2.0 release, so my question is what should I change to

make opk running in SDK?
I am suspecting I need to replace opkg-cl with opkg, something more is 
needed?


Regards,
Marcin
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

For what it's worth, this is the offending portion of Makefile.pre:452:

# Create build directory and generate the sysconfig build-time data there.
# pybuilddir.txt contains the name of the build dir and is used for
# sys.path fixup -- see Modules/getpath.c.
# Since this step runs before shared modules are built, try to avoid
bootstrap
# problems by creating a dummy pybuilddir.txt just to allow interpreter
# initialization to succeed.  It will be overwritten by generate-posix-vars
# or removed in case of failure.
pybuilddir.txt: $(BUILDPYTHON)
→ @echo "none" > ./pybuilddir.txt
→ $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars ;\
→ if test $$? -ne 0 ; then \
→ → echo "generate-posix-vars failed" ; \
→ → rm -f ./pybuilddir.txt ; \
→ → exit 1 ; \
→ fi

I don't know enough about the Python build to understand what it's trying
to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
acceptable?

I'm surprised this seems to work for other people, since this should be
failing for anyone using Python on a target platform different from their
host.

On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle 
wrote:

> On 11/24/15 3:23 PM, Mark Hatle wrote:
> > My guess is that there is a bug in the python integration where it's not
> > realizing the host and target are different systems, so it's trying to
> run a
> > target program on your host.  Your host isn't haswell, so... Illegal
> Instruction
> > -- and the system stops.
>
> Just an FYI, I hit the same problem today.  I suspect it is the host
> trying to
> run target software and I'm looking into it.
>
> --Mark
>
> > The alternative would be something is running QEMU to execute a target
> binary
> > and QEMU doesn't have instruction support -- but that doesn't look like
> the case
> > here.
> >
> > --Mark
> >
> > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >> All,
> >>
> >> I added a new machine definition with tuning parameters for haswell
> >> microarchitectures (basically just duplicated corei7 but tuned it for
> haswell).
> >> This seems to work correctly, but failed when running do_install for
> python
> >> toward the end of the build process. I am running with the Yocto 2.0
> framework,
> >> with very minimal changes to create a new distribution and machine for
> our
> >> custom embedded device. Note I had this distro configuration working
> before, and
> >> the only difference is I added a new machine with this tuning.
> >>
> >> I believe the issue is because, as part of the do_install step for
> Python, it
> >> attempts to run python on the local host, despite being cross-compiled.
> However,
> >> it is tuned for a processor that my host machine doesn't run, so I get
> an
> >> instruction exception.
> >>
> >> Did I do something weird? I figure python would be configured for
> >> cross-compiling and therefore wouldn't try to run the target version on
> the
> >> host, even for sanity tests. Here is the output of the failure:
> >>
> >> $ tail -20
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> >>
> >> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
> >>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
> >>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
> >>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
> >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
> >> x86_64-diags-linux-ranlib libpython2.7.a
> >> Modules/posixmodule.o: In function `posix_tmpnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> >> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
> >> Modules/posixmodule.o: In function `posix_tempnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> >> warning: the use of `tempnam' is dangerous, better use `mkstemp'
> >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse
> -mavx2
> >> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
> >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python
> \
> >> Modules/python.o \
> >> -L. -lpython2.7 -lpthread -ldl  -lpthread
> -lutil   -lm
> >>
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> >> _PYTHON_HOST_PLATFORM=linux2-x86_64
> >>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> >> python2.7 -S -m sysconfig --generate-posix-vars ;\
> >> if test $? -ne 0 ; then \
> >> echo "generate-posix-vars failed" ; \
> >> rm 

Re: [yocto] [OE-core] RFC: Reference updater filesystem

[Randy Witt]

On Mon, Nov 23, 2015 at 1:41 PM, Mariano Lopez <
mariano.lo...@linux.intel.com> wrote:

> There has been interest in an image based software updater in Yocto
> Project. The proposed solution for a image based updater is to use Stefano
> Babic's software updater (http://sbabic.github.io/swupdate). This
> software do a binary copy, so it is needed to have at least two partitions,
> these partitions would be the rootfs and the maintenance partition. The
> rootfs it's the main partition used to boot during the normal device
> operation, on the other hand, the maintenance will be used to update the
> main partition.
>
> To update the system, the user has to connect to device and boot in the
> maintenance partition; once in the maintenance partition the software
> updater will copy the new image in the rootfs partition. A final reboot
> into the rootfs it is necessary to complete the upgrade.
>
> As mentioned before the the software will copy an image to the partition,
> so everything in that partition will be wiped out, including custom
> configurations. To avoid the loss of configuration I explore three
> different solutions:
> 1. Use a separate partition for the configuration.
>   a. The pro of this method is the partition is not touched during the
> update.
>   b. The con of this method is the configuration is not directly in rootfs
> (example: /etc).
>
> Configuration files can be anywhere a package decides to install them. So
having a single partition would be difficult. If you could, you would most
likely be forced to have an initramfs to make sure /etc was mounted before
init runs.


> 2. Do the backup during the update.
>   a. The pro is the configuration is directly in rootfs.
>   b. The con is If the update fail most likely the configuration would be
> lost.
>
> Why would the configuration be lost if the update fails? Couldn't it just
be stored on the thumbdrive?


> 3. Have an OverlayFS for the rootfs or the partition that have the
> configuration.
>   a. The pro is the configuration is  "directly" in rootfs.
>   b. The con is there is need to provide a custom init to guarantee the
> Overlay is mounted before the boot process.
>
> With the above information I'm proposing to use a separate partition for
> the configuration; this is because is more reliable and doesn't require big
> changes in the current architecture.
>
> So, the idea is to have 4 partitions in the media:
> 1. boot. This is the usual boot partition
> 2. data. This will hold the configuration files. Not modified by updates.
> 3. maintenance. This partition will be used to update rootfs.
> 4. rootfs. Partition used for normal operation.
>
> Mariano
> --
> ___
> Openembedded-core mailing list
> openembedded-c...@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to
a native version of Python:

*| which python2.7*
*|
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
|
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
python2.7 -S -m sysconfig --generate-posix-vars ;\
|   if test $? -ne 0 ; then \
|   echo "generate-posix-vars failed" ; \
|   rm -f ./pybuilddir.txt ; \
|   exit 1 ; \
|   fi
| Illegal instruction (core dumped)
| make: *** [sharedmods] Error 132

I guess one of the environmental variables being passed in is screwing up
what it's doing. Unfortunately I don't know enough about the inner workings
of Python to be of much help moving forward. I gave it my best shot!


On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi 
wrote:

> Ah sorry, I misspoke. I walked through the Makefile and configure scripts
> a bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> Python. It seems it's not being configured correctly. Either the path is
> setup wrong and python2.7 is not pointing to the right version, or
> python2.7 needs to be pointing to the absolute path instead of relying on
> $PATH.
>
>
>
>
> On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi 
> wrote:
>
>> For what it's worth, this is the offending portion of Makefile.pre:452:
>>
>> # Create build directory and generate the sysconfig build-time data there.
>> # pybuilddir.txt contains the name of the build dir and is used for
>> # sys.path fixup -- see Modules/getpath.c.
>> # Since this step runs before shared modules are built, try to avoid
>> bootstrap
>> # problems by creating a dummy pybuilddir.txt just to allow interpreter
>> # initialization to succeed.  It will be overwritten by
>> generate-posix-vars
>> # or removed in case of failure.
>> pybuilddir.txt: $(BUILDPYTHON)
>> → @echo "none" > ./pybuilddir.txt
>> → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars
>> ;\
>> → if test $$? -ne 0 ; then \
>> → → echo "generate-posix-vars failed" ; \
>> → → rm -f ./pybuilddir.txt ; \
>> → → exit 1 ; \
>> → fi
>>
>> I don't know enough about the Python build to understand what it's trying
>> to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
>> acceptable?
>>
>> I'm surprised this seems to work for other people, since this should be
>> failing for anyone using Python on a target platform different from their
>> host.
>>
>> On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle 
>> wrote:
>>
>>> On 11/24/15 3:23 PM, Mark Hatle wrote:
>>> > My guess is that there is a bug in the python integration where it's
>>> not
>>> > realizing the host and target are different systems, so it's trying to
>>> run a
>>> > target program on your host.  Your host isn't haswell, so... Illegal
>>> Instruction
>>> > -- and the system stops.
>>>
>>> Just an FYI, I hit the same problem today.  I suspect it is the host
>>> trying to
>>> run target software and I'm looking into it.
>>>
>>> --Mark
>>>
>>> > The alternative would be something is running QEMU to execute a target
>>> binary
>>> > and QEMU doesn't have instruction support -- but that doesn't look
>>> like the case
>>> > here.
>>> >
>>> > --Mark
>>> >
>>> > On 11/24/15 3:06 PM, Michael Habibi wrote:
>>> >> All,
>>> >>
>>> >> I added a new machine definition with tuning parameters for haswell
>>> >> microarchitectures (basically just duplicated corei7 but tuned it for
>>> haswell).
>>> >> This seems to work correctly, but failed when running do_install for
>>> python
>>> >> toward the end of the build process. I am running with the Yocto 2.0
>>> framework,
>>> >> with very minimal changes to create a new distribution and machine
>>> for our
>>> >> custom embedded device. Note I had this distro configuration working
>>> before, and
>>> >> the only difference is I added a new machine with this tuning.
>>> >>
>>> >> I believe the issue is because, as part of the do_install step for
>>> Python, it
>>> >> attempts to run python on the local host, despite being
>>> cross-compiled. However,
>>> >> it is tuned for a processor that my host machine doesn't run, so I
>>> get an
>>> >> instruction exception.
>>> >>
>>> >> Did I do something weird? I figure python would be configured for
>>> >> cross-compiling and therefore wouldn't try to run the target version
>>> on the
>>> >> host, even for sanity tests. Here is the output of the failure:
>>> >>
>>> >> $ tail -20
>>> >>
>>> 

[yocto] [[AUH] 02/17] upgradehelper.py: Merge options into a dictionary

[Aníbal Limón]

Use one dictionary to keep together the options this enables
possibility to define an interface for steps, see next commit.

Signed-off-by: Aníbal Limón 
---
 upgradehelper.py | 83 
 1 file changed, 42 insertions(+), 41 deletions(-)

diff --git a/upgradehelper.py b/upgradehelper.py
index 128bc07..1c3dcbc 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -126,15 +126,38 @@ def parse_config_file(config_file):
 
 class Updater(object):
 def __init__(self, auto_mode=False, send_email=False, 
skip_compilation=False):
+build_dir = get_build_dir()
 
-self.uh_dir = os.path.join(get_build_dir(), "upgrade-helper")
+self.bb = Bitbake(build_dir)
+
+try:
+self.base_env = self.bb.env()
+except EmptyEnvError as e:
+import traceback
+E( " %s\n%s" % (e.message, traceback.format_exc()))
+E( " Bitbake output:\n%s" % (e.stdout))
+exit(1)
+
+self.email_handler = Email(settings)
+self.statistics = Statistics()
+self.git = None
+
+self.opts = {}
+self.opts['interactive'] = not auto_mode
+self.opts['send_email'] = send_email
+self.opts['author'] = "Upgrade Helper <%s>" % \
+settings.get('from', 'u...@not.set')
+self.opts['machines'] = settings.get('machines',
+'qemux86 qemux86-64 qemuarm qemumips qemuppc').split()
+self.opts['skip_compilation'] = skip_compilation
+self.opts['buildhistory_enabled'] = self._buildhistory_is_enabled()
+
+self.uh_dir = os.path.join(build_dir, "upgrade-helper")
 if not os.path.exists(self.uh_dir):
 os.mkdir(self.uh_dir)
-
 self.uh_work_dir = os.path.join(self.uh_dir, "work-%s" % \
 datetime.now().strftime("%Y%m%d%H%M%S"))
 os.mkdir(self.uh_work_dir)
-
 self.uh_recipes_all_dir = os.path.join(self.uh_work_dir, "all")
 os.mkdir(self.uh_recipes_all_dir)
 self.uh_recipes_succeed_dir = os.path.join(self.uh_work_dir, "succeed")
@@ -142,16 +165,6 @@ class Updater(object):
 self.uh_recipes_failed_dir = os.path.join(self.uh_work_dir, "failed")
 os.mkdir(self.uh_recipes_failed_dir)
 
-self.bb = Bitbake(get_build_dir())
-self.git = None
-self.author_email = settings.get('from', 'u...@not.set')
-self.author = "Upgrade Helper <%s>" % self.author_email
-self.skip_compilation = skip_compilation
-self.interactive = not auto_mode
-self.send_email = send_email
-
-self.machines = settings.get('machines', 'qemux86 qemux86-64 qemuarm 
qemumips qemuppc').split()
-
 self.upgrade_steps = [
 (self._load_env, "Loading environment ..."),
 (self._create_workdir, None),
@@ -167,18 +180,6 @@ class Updater(object):
 (self._buildhistory_diff, None)
 ]
 
-try:
-self.base_env = self.bb.env()
-except EmptyEnvError as e:
-import traceback
-E( " %s\n%s" % (e.message, traceback.format_exc()))
-E( " Bitbake output:\n%s" % (e.stdout))
-exit(1)
-self.buildhistory_enabled = self._buildhistory_is_enabled()
-
-self.email_handler = Email(settings)
-self.statistics = Statistics()
-
 def _get_status_msg(self, err):
 if err:
 return str(err)
@@ -199,7 +200,7 @@ class Updater(object):
 " BUILDHISTORY_COMMIT=1 please set.")
 exit(1)
 
-if self.skip_compilation:
+if self.opts['skip_compilation']:
 W(" Buildhistory disabled because user" \
 " skip compilation!")
 else:
@@ -228,7 +229,7 @@ class Updater(object):
 
 stdout = self.git.status()
 if stdout != "":
-if self.interactive:
+if self.opts['interactive']:
 W(" %s: git repository has uncommited work which will be 
dropped! Proceed? (y/N)" % self.pn)
 answer = sys.stdin.readline().strip().upper()
 if answer == '' or answer != 'Y':
@@ -261,16 +262,16 @@ class Updater(object):
 else:
 raise UnsupportedProtocolError
 
-self.recipe = recipe(self.env, self.new_ver, self.interactive, 
self.workdir,
+self.recipe = recipe(self.env, self.new_ver, self.opts['interactive'], 
self.workdir,
  self.recipe_dir, self.bb, self.git)
 
 def _buildhistory_init(self):
-if self.buildhistory_enabled == False:
+if not self.opts['buildhistory_enabled']:
 return
 
 self.buildhistory = BuildHistory(self.bb, self.pn, self.workdir)
-I(" %s: Initial buildhistory for %s ..." % (self.pn, self.machines))
-self.buildhistory.init(self.machines)
+I(" %s: Initial buildhistory 

[yocto] [[AUH] 01/17] {upgradehelper, bitbake}.py: Move _get_env function to bitbake

[Aníbal Limón]

Remove unnecesary access function (_get_env) to environment move
logic to build dictionary to bitbake env method.

Signed-off-by: Aníbal Limón 
---
 bitbake.py   | 19 ++-
 upgradehelper.py | 26 --
 2 files changed, 22 insertions(+), 23 deletions(-)

diff --git a/bitbake.py b/bitbake.py
index a1587ce..cdbce2b 100644
--- a/bitbake.py
+++ b/bitbake.py
@@ -29,6 +29,8 @@ from logging import debug as D
 from logging import error as E
 from logging import critical as C
 import sys
+import re
+
 from errors import *
 
 for path in os.environ["PATH"].split(':'):
@@ -80,7 +82,22 @@ class Bitbake(object):
 return os.path.join(self.log_dir, BITBAKE_ERROR_LOG)
 
 def env(self, recipe=None):
-return self._cmd(recipe, "-e", output_filter="-v \"^#\"")
+stdout = self._cmd(recipe, "-e", output_filter="-v \"^#\"")
+
+assignment = re.compile("^([^ \t=]*)=(.*)")
+bb_env = dict()
+for line in stdout.split('\n'):
+m = assignment.match(line)
+if m:
+if m.group(1) in bb_env:
+continue
+
+bb_env[m.group(1)] = m.group(2).strip("\"")
+
+if not bb_env:
+raise EmptyEnvError(stdout)
+
+return bb_env
 
 def fetch(self, recipe):
 return self._cmd(recipe, "-c fetch")
diff --git a/upgradehelper.py b/upgradehelper.py
index 0223ac0..128bc07 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -168,7 +168,7 @@ class Updater(object):
 ]
 
 try:
-self.base_env = self._get_env()
+self.base_env = self.bb.env()
 except EmptyEnvError as e:
 import traceback
 E( " %s\n%s" % (e.message, traceback.format_exc()))
@@ -185,24 +185,6 @@ class Updater(object):
 else:
 return "Succeeded"
 
-def _get_env(self, pn=None):
-stdout = self.bb.env(pn)
-
-assignment = re.compile("^([^ \t=]*)=(.*)")
-bb_env = dict()
-for line in stdout.split('\n'):
-m = assignment.match(line)
-if m:
-if m.group(1) in bb_env:
-continue
-
-bb_env[m.group(1)] = m.group(2).strip("\"")
-
-if not bb_env:
-raise EmptyEnvError(stdout)
-
-return bb_env
-
 def _buildhistory_is_enabled(self):
 enabled = False
 
@@ -226,7 +208,7 @@ class Updater(object):
 return enabled
 
 def _load_env(self):
-self.env = self._get_env(self.pn)
+self.env = self.bb.env(self.pn)
 
 def _create_workdir(self):
 self.workdir = os.path.join(self.uh_recipes_all_dir, self.pn)
@@ -257,7 +239,7 @@ class Updater(object):
 self.git.reset_hard()
 self.git.clean_untracked()
 
-self.env = self._get_env(self.pn)
+self.env = self.bb.env(self.pn)
 
 def _clean_repo(self):
 try:
@@ -296,7 +278,7 @@ class Updater(object):
 def _rename(self):
 self.recipe.rename()
 
-self.env = self._get_env(self.pn)
+self.env = self.bb.env(self.pn)
 
 self.recipe.update_env(self.env)
 
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 05/17] buildhistory: Add option for enable in upgrade-helper.conf

[Aníbal Limón]

Make consistent the enablement for this feature it need to be
explicit enable into upgrade-helper.conf and also needs configuration
in local.conf.

Signed-off-by: Aníbal Limón 
---
 README   |  9 +++--
 modules/steps.py |  6 +++---
 upgradehelper.py | 40 ++--
 3 files changed, 36 insertions(+), 19 deletions(-)

diff --git a/README b/README
index 44c3bf1..ee1bd1f 100644
--- a/README
+++ b/README
@@ -58,6 +58,10 @@ drop_previous_commits=yes
 
 # machines to test build with
 machines=qemux86 qemux86-64 qemuarm qemumips qemuppc
+
+# optional features
+buildhistory=no
+
 --- snip ---
 
 3. Enable distrodata and supply appropriate additional metadata. For
@@ -75,8 +79,9 @@ WARNING: if you are using the default maintainers.inc file 
supplied
  emails to the default maintainers. Please be careful not to
  do this :)
 
-4. Enable buildhistory by adding the following lines to your
-conf/local.conf file:
+4. If you want to enable buildhistory (optional) you need to enable in
+upgrade-helper.conf also add the following lines to your conf/local.conf
+file:
 
 --- snip ---
 INHERIT =+ "buildhistory"
diff --git a/modules/steps.py b/modules/steps.py
index b85a0dc..4a2eee3 100644
--- a/modules/steps.py
+++ b/modules/steps.py
@@ -85,7 +85,7 @@ def detect_recipe_type(bb, git, opts, pkg_ctx):
 pkg_ctx['recipe_dir'], bb, git)
 
 def buildhistory_init(bb, git, opts, pkg_ctx):
-if not opts['buildhistory_enabled']:
+if not opts['buildhistory']:
 return
 
 pkg_ctx['buildhistory'] = BuildHistory(bb, pkg_ctx['PN'],
@@ -118,11 +118,11 @@ def compile(bb, git, opts, pkg_ctx):
 for machine in opts['machines']:
 I(" %s: compiling for %s ..." % (pkg_ctx['PN'], machine))
 pkg_ctx['recipe'].compile(machine)
-if opts['buildhistory_enabled']:
+if opts['buildhistory']:
 pkg_ctx['buildhistory'].add()
 
 def buildhistory_diff(bb, git, opts, pkg_ctx):
-if not opts['buildhistory_enabled']:
+if not opts['buildhistory']:
 return
 
 I(" %s: Checking buildhistory ..." % pkg_ctx['PN'])
diff --git a/upgradehelper.py b/upgradehelper.py
index e526597..c841c2a 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -156,7 +156,7 @@ class Updater(object):
 self.opts['machines'] = settings.get('machines',
 'qemux86 qemux86-64 qemuarm qemumips qemuppc').split()
 self.opts['skip_compilation'] = skip_compilation
-self.opts['buildhistory_enabled'] = self._buildhistory_is_enabled()
+self.opts['buildhistory'] = self._buildhistory_is_enabled()
 
 self.uh_dir = os.path.join(build_dir, "upgrade-helper")
 if not os.path.exists(self.uh_dir):
@@ -180,22 +180,34 @@ class Updater(object):
 def _buildhistory_is_enabled(self):
 enabled = False
 
-if 'buildhistory' in self.base_env['INHERIT']:
-if not 'BUILDHISTORY_COMMIT' in self.base_env:
-E(" Buildhistory was enabled but need"\
-" BUILDHISTORY_COMMIT=1 please set.")
-exit(1)
+if settings.get("buildhistory", "no") == "yes":
+if 'buildhistory' in self.base_env['INHERIT']:
+if not 'BUILDHISTORY_COMMIT' in self.base_env:
+E(" Buildhistory was INHERIT in conf/local.conf"\
+  " but need BUILDHISTORY_COMMIT=1 please set.")
+exit(1)
 
-if not self.base_env['BUILDHISTORY_COMMIT'] == '1':
-E(" Buildhistory was enabled but need"\
-" BUILDHISTORY_COMMIT=1 please set.")
-exit(1)
+if not self.base_env['BUILDHISTORY_COMMIT'] == '1':
+E(" Buildhistory was INHERIT in conf/local.conf"\
+  " but need BUILDHISTORY_COMMIT=1 please set.")
+exit(1)
 
-if self.opts['skip_compilation']:
-W(" Buildhistory disabled because user" \
-" skip compilation!")
+if self.opts['skip_compilation']:
+W(" Buildhistory disabled because user" \
+" skip compilation!")
+else:
+enabled = True
 else:
-enabled = True
+E(" Buildhistory was enabled in upgrade-helper.conf"\
+  " but isn't INHERIT in conf/local.conf, if you want"\
+  " to enable please set.")
+exit(1)
+else:
+if 'buildhistory' in self.base_env['INHERIT']:
+E(" Buildhistory was INHERIT in conf/local.conf"\
+  " but buildhistory=yes isn't in upgrade-helper.conf,"\
+  " if you want to enable please set.")
+exit(1)
 
 return enabled
 
-- 
2.1.4

-- 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Mark,

I ran the same command, including the offending PYTHONPATH, from the shell
without error. Any reason why that would work, despite it not working from
within the Makefile? I was thinking it was another environment variable
that was causing the issue (one that's not set in my bash environment by
default). My $PATH didn't have python-native so I called it using an
absolute path, but otherwise it's the same command run from the Makefile
(parsed, of course):

bash$
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
-S -m sysconfig --generate-posix-vars
bash$ ls pybuilddir.txt
pybuilddir.txt

Maybe I'm missing something.


On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle 
wrote:

> On 11/25/15 10:11 AM, Michael Habibi wrote:
> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing
> to a
> > native version of Python:
> >
> > *| which python2.7*
> > *|
> >
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> > |
> >
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> > _PYTHON_HOST_PLATFORM=linux2-x86_64
> >
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> > python2.7 -S -m sysconfig --generate-posix-vars ;\
> > |   if test $? -ne 0 ; then \
> > |   echo "generate-posix-vars failed" ; \
> > |   rm -f ./pybuilddir.txt ; \
> > |   exit 1 ; \
> > |   fi
> > | Illegal instruction (core dumped)
> > | make: *** [sharedmods] Error 132
> >
> > I guess one of the environmental variables being passed in is screwing
> up what
> > it's doing. Unfortunately I don't know enough about the inner workings
> of Python
> > to be of much help moving forward. I gave it my best shot!
>
> The problem I tracked down was:
>
>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>
> This is full of .so objects when loaded cause the illegal instruction.
>
> I came up with a patch that I thought was going to fix it, but it's
> triggered
> other failures.
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>
> Just to be clear.. this does NOT work properly in many cases.  But might
> give
> you or someone else a clue as to how to possibly fix it.
>
> --Mark
>
> >
> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi  > > wrote:
> >
> > Ah sorry, I misspoke. I walked through the Makefile and configure
> scripts a
> > bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> > Python. It seems it's not being configured correctly. Either the
> path is
> > setup wrong and python2.7 is not pointing to the right version, or
> python2.7
> > needs to be pointing to the absolute path instead of relying on
> $PATH.
> >
> >
> >
> >
> > On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <
> mikehab...@gmail.com
> > > wrote:
> >
> > For what it's worth, this is the offending portion of
> Makefile.pre:452:
> >
> > # Create build directory and generate the sysconfig build-time
> data there.
> > # pybuilddir.txt contains the name of the build dir and is used
> for
> > # sys.path fixup -- see Modules/getpath.c.
> > # Since this step runs before shared modules are built, try to
> avoid
> > bootstrap
> > # problems by creating a dummy pybuilddir.txt just to allow
> interpreter
> > # initialization to succeed.  It will be overwritten by
> generate-posix-vars
> > # or removed in case of failure.
> > pybuilddir.txt: $(BUILDPYTHON)
> > → @echo "none" > ./pybuilddir.txt
> > → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig
> --generate-posix-vars ;\
> > → if test $$? -ne 0 ; then \
> > → → echo "generate-posix-vars failed" ; \
> > → → rm -f ./pybuilddir.txt ; \
> > → → exit 1 ; \
> > → fi
> >
> > I don't know enough about the Python build to understand what
> it's
> > trying to do, but perhaps replacing PYTHON_FOR_BUILD with
> HOSTPYTHON may
> > be acceptable?
> >
> > I'm surprised this 

[yocto] [[AUH] 03/17] upgradehelper.py: Adds own module for steps

[Aníbal Limón]

Isolate steps into own module in order to provide better
structure of the code.

In order to do that the step interfaces was defined to,
step(bb, git, opts, pkg_ctx)

- bb: Bitbake helper instance.
- git: Git helper instance.
- opts: Upgrade helper options.
- pkg_ctx: Package context per upgrade, it stores
all information related to one upgrade.

Signed-off-by: Aníbal Limón 
---
 steps.py | 142 ++
 upgradehelper.py | 259 +--
 2 files changed, 221 insertions(+), 180 deletions(-)
 create mode 100644 steps.py

diff --git a/steps.py b/steps.py
new file mode 100644
index 000..ea314da
--- /dev/null
+++ b/steps.py
@@ -0,0 +1,142 @@
+#!/usr/bin/env python
+# vim: set ts=4 sw=4 et:
+#
+# Copyright (c) 2013 - 2015 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, 
USA.
+#
+
+import os
+import sys
+
+from logging import debug as D
+from logging import info as I
+from logging import warning as W
+from logging import error as E
+from logging import critical as C
+
+from errors import *
+from buildhistory import BuildHistory
+from recipe import Recipe
+from gitrecipe import GitRecipe
+from svnrecipe import SvnRecipe
+
+def load_env(bb, git, opts, pkg_ctx):
+stdout = git.status()
+if stdout != "":
+if opts['interactive']:
+W(" %s: git repository has uncommited work which will be dropped!" 
\
+" Proceed? (y/N)" % pkg_ctx['PN'])
+answer = sys.stdin.readline().strip().upper()
+if answer == '' or answer != 'Y':
+I(" %s: User abort!" % pkg_ctx['PN'])
+exit(1)
+
+I(" %s: Dropping uncommited work!" % pkg_ctx['PN'])
+git.reset_hard()
+git.clean_untracked()
+
+pkg_ctx['env'] = bb.env(pkg_ctx['PN'])
+if pkg_ctx['env']['PV'] == pkg_ctx['NPV']:
+raise UpgradeNotNeededError
+
+def load_dirs(bb, git, opts, pkg_ctx):
+pkg_ctx['workdir'] = os.path.join(pkg_ctx['base_dir'], pkg_ctx['PN'])
+os.mkdir(pkg_ctx['workdir'])
+
+pkg_ctx['recipe_dir'] = os.path.dirname(pkg_ctx['env']['FILE'])
+
+def clean_repo(bb, git, opts, pkg_ctx):
+try:
+git.checkout_branch("upgrades")
+except Error:
+git.create_branch("upgrades")
+
+try:
+git.delete_branch("remove_patches")
+except:
+pass
+
+def detect_recipe_type(bb, git, opts, pkg_ctx):
+if pkg_ctx['env']['SRC_URI'].find("ftp://;) != -1 or  \
+pkg_ctx['env']['SRC_URI'].find("http://;) != -1 or \
+pkg_ctx['env']['SRC_URI'].find("https://;) != -1:
+recipe = Recipe
+elif pkg_ctx['env']['SRC_URI'].find("git://") != -1:
+recipe = GitRecipe
+else:
+raise UnsupportedProtocolError
+
+pkg_ctx['recipe'] = recipe(pkg_ctx['env'], pkg_ctx['NPV'],
+opts['interactive'], pkg_ctx['workdir'],
+pkg_ctx['recipe_dir'], bb, git)
+
+def buildhistory_init(bb, git, opts, pkg_ctx):
+if not opts['buildhistory_enabled']:
+return
+
+pkg_ctx['buildhistory'] = BuildHistory(bb, pkg_ctx['PN'],
+pkg_ctx['workdir'])
+I(" %s: Initial buildhistory for %s ..." % (pkg_ctx['PN'],
+opts['machines']))
+pkg_ctx['buildhistory'].init(opts['machines'])
+
+def unpack_original(bb, git, opts, pkg_ctx):
+pkg_ctx['recipe'].unpack()
+
+def rename(bb, git, opts, pkg_ctx):
+pkg_ctx['recipe'].rename()
+
+pkg_ctx['env'] = bb.env(pkg_ctx['PN'])
+
+pkg_ctx['recipe'].update_env(pkg_ctx['env'])
+
+def cleanall(bb, git, opts, pkg_ctx):
+pkg_ctx['recipe'].cleanall()
+
+def fetch(bb, git, opts, pkg_ctx):
+pkg_ctx['recipe'].fetch()
+
+def compile(bb, git, opts, pkg_ctx):
+if opts['skip_compilation']:
+W(" %s: Compilation was skipped by user choice!")
+return
+
+for machine in opts['machines']:
+I(" %s: compiling for %s ..." % (pkg_ctx['PN'], machine))
+pkg_ctx['recipe'].compile(machine)
+if opts['buildhistory_enabled']:
+pkg_ctx['buildhistory'].add()
+
+def buildhistory_diff(bb, git, opts, pkg_ctx):
+if not opts['buildhistory_enabled']:
+return
+
+I(" %s: Checking buildhistory ..." % pkg_ctx['PN'])
+

[yocto] [[AUH] 06/17] recipe/base.py: Add is_recipe_or_include_file func

[Aníbal Limón]

Instead of have duplicate code add this new function
for make code easy to read.

Signed-off-by: Aníbal Limón 
---
 modules/recipe/base.py | 35 +--
 1 file changed, 17 insertions(+), 18 deletions(-)

diff --git a/modules/recipe/base.py b/modules/recipe/base.py
index 14aa5bb..15c5f43 100644
--- a/modules/recipe/base.py
+++ b/modules/recipe/base.py
@@ -33,6 +33,18 @@ from logging import warning as W
 from errors import *
 from utils.bitbake import *
 
+def is_recipe_or_include_file(full_path_f, f):
+is_file = os.path.isfile(full_path_f)
+
+is_recipe = f.find(self.env['PN']) == 0 and \
+f.find(self.env['PKGV']) != -1 and \
+f.find(".bb") != -1
+
+is_include = f.find(self.env['PN']) == 0 and \
+ f.find(".inc") != -1
+
+return is_file and (is_recipe or is_include)
+
 class Recipe(object):
 def __init__(self, env, new_ver, interactive, workdir, recipe_dir, 
bitbake, git):
 self.env = env
@@ -63,6 +75,7 @@ class Recipe(object):
 
 super(Recipe, self).__init__()
 
+
 def update_env(self, env):
 self.env = env
 
@@ -80,10 +93,7 @@ class Recipe(object):
 # change PR before renaming
 for f in os.listdir(self.recipe_dir):
 full_path_f = os.path.join(self.recipe_dir, f)
-if os.path.isfile(full_path_f) and \
-((f.find(self.env['PN']) == 0 and f.find(self.env['PKGV']) 
!= -1 and
-  f.find(".bb") != -1) or
- (f.find(self.env['PN']) == 0 and f.find(".inc") != -1)):
+if is_recipe_or_include_file(full_path_f, f):
 with open(full_path_f + ".tmp", "w+") as temp_recipe:
 with open(full_path_f) as recipe:
 for line in recipe:
@@ -130,12 +140,7 @@ class Recipe(object):
 
 for f in os.listdir(self.recipe_dir):
 full_path_f = os.path.join(self.recipe_dir, f)
-if os.path.isfile(full_path_f) and \
-((f.find(self.env['PN']) == 0 and
-  f.find(self.env['PKGV']) != -1 and
-  f.find(".bb") != -1) or
- (f.find(self.env['PN']) == 0 and
-  f.find(".inc") != -1)):
+if is_recipe_or_include_file(full_path_f, f):
 with open(full_path_f + ".tmp", "w+") as temp_recipe:
 with open(full_path_f) as recipe:
 for line in recipe:
@@ -178,10 +183,7 @@ class Recipe(object):
 # files... Go through the recipes/inc files until we find them
 for f in os.listdir(self.recipe_dir):
 full_path_f = os.path.join(self.recipe_dir, f)
-if os.path.isfile(full_path_f) and \
-((f.find(self.env['PN']) == 0 and f.find(self.env['PKGV']) 
!= -1 and
-  f.find(".bb") != -1) or
- (f.find(self.env['PN']) == 0 and f.find(".inc") != -1)):
+if is_recipe_or_include_file(full_path_f, f):
 with open(full_path_f + ".tmp", "w+") as temp_recipe:
 with open(full_path_f) as recipe:
 for line in recipe:
@@ -218,10 +220,7 @@ class Recipe(object):
 # Will change the extension of the archive from the SRC_URI
 for f in os.listdir(self.recipe_dir):
 full_path_f = os.path.join(self.recipe_dir, f)
-if os.path.isfile(full_path_f) and \
-((f.find(self.env['PN']) == 0 and f.find(self.env['PKGV']) 
!= -1 and
-  f.find(".bb") != -1) or
- (f.find(self.env['PN']) == 0 and f.find(".inc") != -1)):
+if is_recipe_or_include_file(full_path_f, f):
 with open(full_path_f + ".tmp", "w+") as temp_recipe:
 with open(full_path_f) as recipe:
 source_found = False
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 13/17] upgradehelper: Add testimage feature.

[Aníbal Limón]

Testimage module implements usage of testimage class, a integration
branch is created with succesful recipe upgrades and then the next
tests are run for every machine configured,

- ptest: Recipes that support ptest are run and retrive
 the result after upgrade.
- sato: Runs core-image-sato -c testimage.

The results are stored into recipe work directory and send to the
maintainer.

[YOCTO #7471]
[YOCTO #7567]

Signed-off-by: Aníbal Limón 
---
 modules/testimage.py | 185 +++
 upgradehelper.py |  82 +--
 2 files changed, 261 insertions(+), 6 deletions(-)
 create mode 100644 modules/testimage.py

diff --git a/modules/testimage.py b/modules/testimage.py
new file mode 100644
index 000..3fbbc19
--- /dev/null
+++ b/modules/testimage.py
@@ -0,0 +1,185 @@
+#!/usr/bin/env python
+# vim: set ts=4 sw=4 et:
+#
+# Copyright (c) 2015 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, 
USA.
+#
+# This module implements logic for run image tests on recipes when upgrade
+# process succeed.
+#
+
+import os
+import sys
+import shutil
+
+import logging as log
+from logging import debug as D
+from logging import info as I
+from logging import warning as W
+from logging import error as E
+from logging import critical as C
+
+from errors import *
+from utils.bitbake import *
+
+class TestImage():
+def __init__(self, bb, git, uh_work_dir, pkgs_ctx):
+self.bb = bb
+self.git = git
+self.uh_work_dir = uh_work_dir
+self.pkgs_ctx = pkgs_ctx
+
+os.environ['BB_ENV_EXTRAWHITE'] = os.environ['BB_ENV_EXTRAWHITE'] + \
+" TEST_SUITES CORE_IMAGE_EXTRA_INSTALL"
+
+def _get_ptest_pkgs(self):
+pkgs = []
+
+for c in self.pkgs_ctx:
+if "ptest" in c['recipe'].get_inherits():
+pkgs.append(c)
+
+return pkgs
+
+def _get_pkgs_to_install(self, pkgs, ptest=False):
+pkgs_out = []
+
+# for provide access to the target
+if ptest:
+pkgs_out.append("dropbear")
+
+for c in pkgs:
+pkgs_out.append(c['PN'])
+if ptest:
+pkgs_out.append("%s-ptest" % c['PN'])
+
+return ' '.join(pkgs_out)
+
+def prepare_branch(self):
+self.git.checkout_branch("master")
+try:
+self.git.delete_branch("testimage")
+self.git.delete_branch("upgrades")
+except Error:
+pass
+self.git.reset_hard()
+
+self.git.create_branch("testimage")
+for c in self.pkgs_ctx:
+patch_file = os.path.join(c['workdir'], c['patch_file'])
+self.git.apply_patch(patch_file)
+
+def _parse_ptest_log(self, log_file):
+ptest_results = {}
+
+with open(log_file, "r") as f:
+pn = None
+processing = False
+
+for line in f:
+if not processing:
+m = re.search("^BEGIN: /usr/lib/(.*)/ptest$", line)
+if m:
+pn = m.group(1)
+ptest_results[pn] = []
+processing = True
+else:
+m = re.search("^END: $", line)
+if m:
+pn = None
+processing = False
+else:
+ptest_results[pn].append(line)
+
+return ptest_results
+
+def _find_log(self, name, machine):
+result = []
+
+base_dir = os.path.join(os.getenv('BUILDDIR'), 'tmp', 'work')
+for root, dirs, files in os.walk(base_dir):
+if name in files:
+result.append(os.path.join(root, name))
+
+for ptest_log in result:
+if machine in ptest_log:
+return ptest_log
+
+def ptest(self, machine):
+ptest_pkgs = self._get_ptest_pkgs()
+
+os.environ['CORE_IMAGE_EXTRA_INSTALL'] = \
+self._get_pkgs_to_install(ptest_pkgs, True)
+I( "   building core-image-minimal for %s ..." % machine)
+self.bb.complete("core-image-minimal", machine)
+
+os.environ['TEST_SUITES'] = "ping ssh _ptest"
+I( "   running 

[yocto] [[AUH] 11/17] utils/git.py: Add method for apply patches into a branch.

[Aníbal Limón]

Signed-off-by: Aníbal Limón 
---
 modules/utils/git.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/modules/utils/git.py b/modules/utils/git.py
index 48ed46d..d7f5713 100644
--- a/modules/utils/git.py
+++ b/modules/utils/git.py
@@ -61,6 +61,9 @@ class Git(object):
 def create_patch(self, out_dir):
 return self._cmd("format-patch -M10 -1 -o " + out_dir)
 
+def apply_patch(self, patch_file):
+return self._cmd("am %s" % patch_file)
+
 def status(self):
 return self._cmd("status --porcelain")
 
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 12/17] upgradehelper.py: Add settings for enable testimage.

[Aníbal Limón]

Testimage feature will provide the ability to run testimage
class over a set of succesful upgrades.

Also it will run ptest on recipes that support ptest and retrive
the result for store/send to maintainer.

Signed-off-by: Aníbal Limón 
---
 README   | 25 +
 upgradehelper.py | 38 --
 2 files changed, 61 insertions(+), 2 deletions(-)

diff --git a/README b/README
index ee1bd1f..6432f97 100644
--- a/README
+++ b/README
@@ -61,6 +61,7 @@ machines=qemux86 qemux86-64 qemuarm qemumips qemuppc
 
 # optional features
 buildhistory=no
+testimage=no
 
 --- snip ---
 
@@ -93,6 +94,30 @@ $BUILDDIR/upgrade-helper/work/recipe/buildhistory-diff.txt.
 
 (Do not remove any other inherited class in the process, e.g. distrodata).
 
+5. If you want to enable testimage (optional) you need to enable in
+upgrade-helper.conf also add the following lines to your conf/local.conf
+file:
+
+--- snip ---
+INHERIT += "testimage"
+
+EXTRA_IMAGE_FEATURES = "debug-tweaks package-management"
+# testimage/ptest only work with rpm
+PACKAGE_CLASSES = "package_rpm"
+--- snip ---
+
+Also if you are running in a server without X11 session, you need to start
+a vncserver example:
+
+$ vncserver :1
+$ export DISPLAY=:1
+
+If upgrade is succesful testimage/ptest results are generated into
+$BUILDDIR/upgrade-helper/work/recipe/ptest_recipe.log if recipe support
+ptest.
+
+(Do not remove any other inherited class in the process, e.g. distrodata).
+
 Usage
 -
 
diff --git a/upgradehelper.py b/upgradehelper.py
index c841c2a..222909b 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -157,6 +157,7 @@ class Updater(object):
 'qemux86 qemux86-64 qemuarm qemumips qemuppc').split()
 self.opts['skip_compilation'] = skip_compilation
 self.opts['buildhistory'] = self._buildhistory_is_enabled()
+self.opts['testimage'] = self._testimage_is_enabled()
 
 self.uh_dir = os.path.join(build_dir, "upgrade-helper")
 if not os.path.exists(self.uh_dir):
@@ -211,6 +212,41 @@ class Updater(object):
 
 return enabled
 
+def _testimage_is_enabled(self):
+enabled = False
+
+if settings.get("testimage", "no") == "yes":
+if 'testimage' in self.base_env['INHERIT']:
+if not "ptest" in self.base_env["DISTRO_FEATURES"]:
+E(" testimage requires ptest in DISTRO_FEATURES please add 
to"\
+  " conf/local.conf.")
+exit(1)
+
+if not "package-management" in 
self.base_env['EXTRA_IMAGE_FEATURES']:
+E(" testimage requires package-management in 
EXTRA_IMAGE_FEATURES"\
+  " please add to conf/local.conf.")
+exit(1)
+
+if not "package_rpm" == self.base_env["PACKAGE_CLASSES"]:
+E(" testimage/ptest requires PACKAGE_CLASSES set to 
package_rpm"\
+  " please add to conf/local.conf.")
+exit(1)
+
+enabled = True
+else:
+E(" testimage was enabled in upgrade-helper.conf"\
+  " but isn't INHERIT in conf/local.conf, if you want"\
+  " to enable please set.")
+exit(1)
+else:
+if 'testimage' in self.base_env['INHERIT']:
+E(" testimage was INHERIT in conf/local.conf"\
+  " but testimage=yes isn't in upgrade-helper.conf,"\
+  " if you want to enable please set.")
+exit(1)
+
+return enabled
+
 def _get_packages_to_upgrade(self, packages=None):
 if packages is None:
 I( "Nothing to upgrade")
@@ -497,10 +533,8 @@ class Updater(object):
 self.uh_recipes_failed_dir, pkg_ctx['PN']))
 
 self.commit_changes(pkg_ctx)
-
 self.statistics.update(pkg_ctx['PN'], pkg_ctx['NPV'],
 pkg_ctx['MAINTAINER'], pkg_ctx['error'])
-
 self.pkg_upgrade_handler(pkg_ctx)
 
 if attempted_pkgs > 1:
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 10/17] steps.py: Move clean_repo to first step.

[Aníbal Limón]

In order to avoid random errors of uncleaned repository.

Signed-off-by: Aníbal Limón 
---
 modules/steps.py | 28 
 1 file changed, 16 insertions(+), 12 deletions(-)

diff --git a/modules/steps.py b/modules/steps.py
index 19549cc..2685b80 100644
--- a/modules/steps.py
+++ b/modules/steps.py
@@ -34,6 +34,21 @@ from recipe.base import Recipe
 from recipe.git import GitRecipe
 from recipe.svn import SvnRecipe
 
+def clean_repo(bb, git, opts, pkg_ctx):
+git.checkout_branch("master")
+
+try:
+git.delete_branch("remove_patches")
+except:
+pass
+try:
+git.delete_branch("upgrades")
+except:
+pass
+
+git.reset_hard()
+git.create_branch("upgrades")
+
 def load_env(bb, git, opts, pkg_ctx):
 stdout = git.status()
 if stdout != "":
@@ -57,17 +72,6 @@ def load_env(bb, git, opts, pkg_ctx):
 if pkg_ctx['env']['PV'] == pkg_ctx['NPV']:
 raise UpgradeNotNeededError
 
-def clean_repo(bb, git, opts, pkg_ctx):
-try:
-git.checkout_branch("upgrades")
-except Error:
-git.create_branch("upgrades")
-
-try:
-git.delete_branch("remove_patches")
-except:
-pass
-
 def detect_recipe_type(bb, git, opts, pkg_ctx):
 if pkg_ctx['env']['SRC_URI'].find("ftp://;) != -1 or  \
 pkg_ctx['env']['SRC_URI'].find("http://;) != -1 or \
@@ -127,8 +131,8 @@ def buildhistory_diff(bb, git, opts, pkg_ctx):
 pkg_ctx['buildhistory'].diff()
 
 upgrade_steps = [
-(load_env, "Loading environment ..."),
 (clean_repo, "Cleaning git repository of temporary branch ..."),
+(load_env, "Loading environment ..."),
 (detect_recipe_type, None),
 (buildhistory_init, None),
 (unpack_original, "Fetch & unpack original version ..."),
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 08/17] recipe/base.py: Add support for get recipe inherits.

[Aníbal Limón]

Recipe inherits are needed to detect what recipes support
certain feature.

For example: For detect ptest enable recipes and then do
ptest runtime test after succesful upgrade.

Signed-off-by: Aníbal Limón 
---
 modules/recipe/base.py | 25 +
 1 file changed, 25 insertions(+)

diff --git a/modules/recipe/base.py b/modules/recipe/base.py
index 5c70c61..dabad2f 100644
--- a/modules/recipe/base.py
+++ b/modules/recipe/base.py
@@ -57,6 +57,16 @@ def modify_recipe_files(func):
 os.rename(full_path_f + ".tmp", full_path_f)
 return modify
 
+def read_recipe_files(func):
+def read(env, recipe_dir):
+for f in os.listdir(recipe_dir):
+full_path_f = os.path.join(recipe_dir, f)
+if is_recipe_or_include_file(env, full_path_f, f):
+with open(full_path_f) as recipe:
+for line in recipe:
+func(line)
+return read
+
 class Recipe(object):
 def __init__(self, env, new_ver, interactive, workdir, recipe_dir, 
bitbake, git):
 self.env = env
@@ -85,8 +95,23 @@ class Recipe(object):
 self.commit_msg = self.env['PN'] + ": upgrade to " + self.new_ver + 
"\n\n"
 self.rm_patches_msg = "\n\nRemoved the following patch(es):\n"
 
+self._inherits = None
+
 super(Recipe, self).__init__()
 
+def get_inherits(self):
+@read_recipe_files
+def _get_inherits(line):
+m = re.search("^inherit (.*)$", line)
+if m:
+tmp = m.group(1).split()
+self._inherits.extend(tmp)
+
+if not self._inherits:
+self._inherits = []
+_get_inherits(self.env, self.recipe_dir)
+
+return self._inherits
 
 def update_env(self, env):
 self.env = env
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 09/17] steps.py: Merge load_dirs step into load_env.

[Aníbal Limón]

Signed-off-by: Aníbal Limón 
---
 modules/steps.py | 9 +++--
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/modules/steps.py b/modules/steps.py
index 4a2eee3..19549cc 100644
--- a/modules/steps.py
+++ b/modules/steps.py
@@ -50,15 +50,13 @@ def load_env(bb, git, opts, pkg_ctx):
 git.clean_untracked()
 
 pkg_ctx['env'] = bb.env(pkg_ctx['PN'])
-if pkg_ctx['env']['PV'] == pkg_ctx['NPV']:
-raise UpgradeNotNeededError
-
-def load_dirs(bb, git, opts, pkg_ctx):
 pkg_ctx['workdir'] = os.path.join(pkg_ctx['base_dir'], pkg_ctx['PN'])
 os.mkdir(pkg_ctx['workdir'])
-
 pkg_ctx['recipe_dir'] = os.path.dirname(pkg_ctx['env']['FILE'])
 
+if pkg_ctx['env']['PV'] == pkg_ctx['NPV']:
+raise UpgradeNotNeededError
+
 def clean_repo(bb, git, opts, pkg_ctx):
 try:
 git.checkout_branch("upgrades")
@@ -130,7 +128,6 @@ def buildhistory_diff(bb, git, opts, pkg_ctx):
 
 upgrade_steps = [
 (load_env, "Loading environment ..."),
-(load_dirs, None),
 (clean_repo, "Cleaning git repository of temporary branch ..."),
 (detect_recipe_type, None),
 (buildhistory_init, None),
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 17/17] statistics: Add support for publish_work_url setting.

[Aníbal Limón]

Publish setting helps to point the url where the AUH patches and
logs are saved, this url is included into AUH upgrade status email.

Signed-off-by: Aníbal Limón 
---
 README|  3 +++
 modules/statistics.py | 10 --
 upgradehelper.py  | 12 ++--
 3 files changed, 17 insertions(+), 8 deletions(-)

diff --git a/README b/README
index d0cc4ae..0756fdc 100644
--- a/README
+++ b/README
@@ -50,6 +50,9 @@ status_recipients=john@doe.com
 # specify the directory where work (patches) will be saved (optional)
 #workdir=
 
+# publish url to use in statistics summary (optional)
+#publish_work_url=http://auh.somehost.com/work
+
 # clean sstate directory before upgrading
 clean_sstate=yes
 
diff --git a/modules/statistics.py b/modules/statistics.py
index 905a4cc..db83c5b 100644
--- a/modules/statistics.py
+++ b/modules/statistics.py
@@ -99,8 +99,14 @@ class Statistics(object):
 
 return stat_msg
 
-def get_summary(self):
-msg = self._pkg_stats()
+def get_summary(self, publish_work_url, workdir):
+msg = ''
+
+if publish_work_url:
+msg += "AUH finished upgrade batch the result patches/logs can be 
found at:\n" \
+   "%s/%s, next are the statistics:\n\n" % (publish_work_url, 
workdir)
+
+msg += self._pkg_stats()
 msg += self._maintainer_stats()
 
 return msg
diff --git a/upgradehelper.py b/upgradehelper.py
index d0e1439..0d3f8b2 100755
--- a/upgradehelper.py
+++ b/upgradehelper.py
@@ -393,7 +393,7 @@ class Updater(object):
 I(" %s: %s" % (pkg_ctx['PN'], e.stdout))
 raise e
 
-def send_status_mail(self):
+def send_status_mail(self, statistics_summary):
 if "status_recipients" not in settings:
 E(" Could not send status email, no recipients set!")
 return -1
@@ -402,10 +402,8 @@ class Updater(object):
 
 subject = "[AUH] Upgrade status: " + date.isoformat(date.today())
 
-msg = self.statistics.get_summary()
-
 if self.statistics.total_attempted:
-self.email_handler.send_email(to_list, subject, msg)
+self.email_handler.send_email(to_list, subject, statistics_summary)
 else:
 W("No recipes attempted, not sending status mail!")
 
@@ -611,7 +609,9 @@ class Updater(object):
 self.pkg_upgrade_handler(pkg_ctx)
 
 if attempted_pkgs > 0:
-statistics_summary = self.statistics.get_summary()
+statistics_summary = self.statistics.get_summary(
+settings.get('publish_work_url', 'no'),
+os.path.basename(self.uh_work_dir))
 
 statistics_file = os.path.join(self.uh_work_dir,
 "statistics_summary")
@@ -621,7 +621,7 @@ class Updater(object):
 I(" %s" % statistics_summary)
 
 if self.opts['send_email']:
-self.send_status_mail()
+self.send_status_mail(statistics_summary)
 
 class UniverseUpdater(Updater):
 def __init__(self):
-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Help with Opensips Recipe - QA run found fatal errors

[Paul Eggleton]

Hi Caio,

On Tuesday 24 November 2015 11:48:13 Caio wrote:
> I'm writing a recipe to a Sip proxy Opensips, but I'm having problems with
> QA Issues.
> In the First time I tried to use make install from Opensips Makefile I had
> problem with QA Issues,  so I added all the folders to FILES_${PN} and
> FILES_${PN} but I still having problem.
> 
> So I decided to start small, creating the folder structure and copy only
> the binary files and after I will copy all the files using the  do_install
> function.
> 
> But when I added the folders and I started to have problems with QA again.
> 
> FILES_${PN} = " \
> ${sbindir}/* \
> ${sysconfdir}/* \
> ${sysconfdir}/opensips/* \
> ${libdir}/* \
> ${libdir}/opensips/* \
> ${libdir}/opensips/modules/* \
> "
> Someone could help me with this problem?
> 
> 
> Thank you very much
> 
> Best Regards,
> 
> Caio Pereira
> 
> 
> ERROR: QA Issue: opensips: The compile log indicates that host include
> and/or library paths were used.
>  Please check the log
> '/home/icchw/fsl-community-bsp/small_steps/tmp/work/armv7at2hf-vfp-neon-poky
> -linux-gnueabi/opensips/2.2.0-r0/temp/log.do_compile' for more information.
> [compile-host-path]
> 
> 
> ERROR: QA run found fatal errors. Please consider fixing them.
> ERROR: Function failed: do_package_qa
> ERROR: Logfile of failure stored in:
> /home/icchw/fsl-community-bsp/small_steps/tmp/work/armv7at2hf-vfp-neon-poky-
> linux-gnueabi/opensips/2.2.0-r0/temp/log.do_package_qa.30973 ERROR: Task 9
> (/home/icchw/fsl-community-bsp/sources/icchw/meta-caio/recipes-telephony/ope
> nsips/ opensips_2.2.0.bb, do_package_qa) failed with exit code '1'
> 
> 
> cat
> /home/icchw/fsl-community-bsp/small_steps/tmp/work/armv7at2hf-vfp-neon-poky-
> linux-gnueabi/opensips/2.2.0-r0/temp/log.do_package_qa.30973
> 
> DEBUG: Python function read_subpackage_metadata finished
> ERROR: QA Issue: opensips: The compile log indicates that host include
> and/or library paths were used.
>  Please check the log
> '/home/icchw/fsl-community-bsp/small_steps/tmp/work/armv7at2hf-vfp-neon-poky
> -linux-gnueabi/opensips/2.2.0-r0/temp/log.do_compile' for more information.
> [compile-host-path]

Did you check the mentioned compile log?

FYI a good reference for these QA checks is:

  
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#qa-errors-and-warnings

As mentioned there you should find somewhere in that log "is unsafe for
cross-compilation" or "CROSS COMPILE Badness".

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Cross-compiling python fails with new custom tuning

[Mark Hatle]

On 11/25/15 10:11 AM, Michael Habibi wrote:
> Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to a
> native version of Python:
> 
> *| which python2.7*
> *|
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> |
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> python2.7 -S -m sysconfig --generate-posix-vars ;\
> |   if test $? -ne 0 ; then \
> |   echo "generate-posix-vars failed" ; \
> |   rm -f ./pybuilddir.txt ; \
> |   exit 1 ; \
> |   fi
> | Illegal instruction (core dumped)
> | make: *** [sharedmods] Error 132
> 
> I guess one of the environmental variables being passed in is screwing up what
> it's doing. Unfortunately I don't know enough about the inner workings of 
> Python
> to be of much help moving forward. I gave it my best shot!

The problem I tracked down was:

PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7

This is full of .so objects when loaded cause the illegal instruction.

I came up with a patch that I thought was going to fix it, but it's triggered
other failures.

http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html

Just to be clear.. this does NOT work properly in many cases.  But might give
you or someone else a clue as to how to possibly fix it.

--Mark

> 
> On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi  > wrote:
> 
> Ah sorry, I misspoke. I walked through the Makefile and configure scripts 
> a
> bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> Python. It seems it's not being configured correctly. Either the path is
> setup wrong and python2.7 is not pointing to the right version, or 
> python2.7
> needs to be pointing to the absolute path instead of relying on $PATH.
> 
> 
> 
> 
> On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi  > wrote:
> 
> For what it's worth, this is the offending portion of 
> Makefile.pre:452:
> 
> # Create build directory and generate the sysconfig build-time data 
> there.
> # pybuilddir.txt contains the name of the build dir and is used for
> # sys.path fixup -- see Modules/getpath.c.
> # Since this step runs before shared modules are built, try to avoid
> bootstrap
> # problems by creating a dummy pybuilddir.txt just to allow 
> interpreter
> # initialization to succeed.  It will be overwritten by 
> generate-posix-vars
> # or removed in case of failure.
> pybuilddir.txt: $(BUILDPYTHON)
> → @echo "none" > ./pybuilddir.txt
> → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig 
> --generate-posix-vars ;\
> → if test $$? -ne 0 ; then \
> → → echo "generate-posix-vars failed" ; \
> → → rm -f ./pybuilddir.txt ; \
> → → exit 1 ; \
> → fi
> 
> I don't know enough about the Python build to understand what it's
> trying to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON 
> may
> be acceptable?
> 
> I'm surprised this seems to work for other people, since this should 
> be
> failing for anyone using Python on a target platform different from
> their host.
> 
> On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle  > wrote:
> 
> On 11/24/15 3:23 PM, Mark Hatle wrote:
> > My guess is that there is a bug in the python integration where
> it's not
> > realizing the host and target are different systems, so it's
> trying to run a
> > target program on your host.  Your host isn't haswell, so...
> Illegal Instruction
> > -- and the system stops.
> 
> Just an FYI, I hit the same problem today.  I suspect it is the 
> host
> trying to
> run target software and I'm looking into it.
> 
> --Mark
> 
> > The alternative would be something is running QEMU to execute a
> target binary
> > and QEMU doesn't have instruction support -- but that doesn't 
> look
> like the case
> > here.
> >
> > --Mark
> >
> > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >> All,
> >>
> >> I added a new machine 

[yocto] [[AUH] 00/17] AUH code refactor and support test image.

[Aníbal Limón]

The following patchset contains,

- A refactor of AUH code clean-up and improve source tree.
- A implementation of testimage that do ptest and sato testimages.
- New settings for publish patches and logs generated by AUH.
- Improvements on the status email.

The changes can also be found at,

http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/log/?h=boot_images_v2

Aníbal Limón (17):
  {upgradehelper, bitbake}.py: Move _get_env function to bitbake
  upgradehelper.py: Merge options into a dictionary
  upgradehelper.py: Adds own module for steps
  upgradehelper: Reorder files into directories.
  buildhistory: Add option for enable in upgrade-helper.conf
  recipe/base.py: Add is_recipe_or_include_file func
  recipe/base.py: Add modify_recipe_files function decorator
  recipe/base.py: Add support for get recipe inherits.
  steps.py: Merge load_dirs step into load_env.
  steps.py: Move clean_repo to first step.
  utils/git.py: Add method for apply patches into a branch.
  upgradehelper.py: Add settings for enable testimage.
  upgradehelper: Add testimage feature.
  upgradehelper.py: Changed retry failure build to 30 days.
  upgradehelper: Add workdir setting.
  statistics: Improve email format and get_summary method.
  statistics: Add support for publish_work_url setting.

 README|  40 ++-
 bitbake.py| 107 ---
 buildhistory.py   |  75 -
 emailhandler.py   | 108 ---
 errors.py |  93 --
 git.py| 101 ---
 gitrecipe.py  |  94 --
 modules/buildhistory.py   |  75 +
 modules/errors.py |  93 ++
 modules/recipe/__init__.py|   0
 modules/recipe/base.py| 682 ++
 modules/recipe/git.py |  94 ++
 modules/recipe/svn.py |  28 ++
 modules/statistics.py | 112 +++
 modules/steps.py  | 144 +
 modules/testimage.py  | 185 
 modules/utils/__init__.py |   0
 modules/utils/bitbake.py  | 124 
 modules/utils/emailhandler.py | 108 +++
 modules/utils/git.py  | 105 +++
 recipe.py | 669 -
 statistics.py | 102 ---
 svnrecipe.py  |  28 --
 upgradehelper.py  | 505 +++
 24 files changed, 2042 insertions(+), 1630 deletions(-)
 delete mode 100644 bitbake.py
 delete mode 100644 buildhistory.py
 delete mode 100644 emailhandler.py
 delete mode 100644 errors.py
 delete mode 100644 git.py
 delete mode 100644 gitrecipe.py
 create mode 100644 modules/buildhistory.py
 create mode 100644 modules/errors.py
 create mode 100644 modules/recipe/__init__.py
 create mode 100644 modules/recipe/base.py
 create mode 100644 modules/recipe/git.py
 create mode 100644 modules/recipe/svn.py
 create mode 100644 modules/statistics.py
 create mode 100644 modules/steps.py
 create mode 100644 modules/testimage.py
 create mode 100644 modules/utils/__init__.py
 create mode 100644 modules/utils/bitbake.py
 create mode 100644 modules/utils/emailhandler.py
 create mode 100644 modules/utils/git.py
 delete mode 100644 recipe.py
 delete mode 100644 statistics.py
 delete mode 100644 svnrecipe.py

-- 
2.1.4

-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] [[AUH] 07/17] recipe/base.py: Add modify_recipe_files function decorator

[Aníbal Limón]

Remove duplicate code adding modify_recipe_files decorator,
this function interate over recipe_dir and found bb and
includes files to make modifications.

Modifications to recipe bb and include files are made by
function passed to modify_recipe_files.

Signed-off-by: Aníbal Limón 
---
 modules/recipe/base.py | 293 +++--
 1 file changed, 140 insertions(+), 153 deletions(-)

diff --git a/modules/recipe/base.py b/modules/recipe/base.py
index 15c5f43..5c70c61 100644
--- a/modules/recipe/base.py
+++ b/modules/recipe/base.py
@@ -33,18 +33,30 @@ from logging import warning as W
 from errors import *
 from utils.bitbake import *
 
-def is_recipe_or_include_file(full_path_f, f):
+def is_recipe_or_include_file(env, full_path_f, f):
 is_file = os.path.isfile(full_path_f)
 
-is_recipe = f.find(self.env['PN']) == 0 and \
-f.find(self.env['PKGV']) != -1 and \
+is_recipe = f.find(env['PN']) == 0 and \
+f.find(env['PKGV']) != -1 and \
 f.find(".bb") != -1
 
-is_include = f.find(self.env['PN']) == 0 and \
+is_include = f.find(env['PN']) == 0 and \
  f.find(".inc") != -1
 
 return is_file and (is_recipe or is_include)
 
+def modify_recipe_files(func):
+def modify(env, recipe_dir):
+for f in os.listdir(recipe_dir):
+full_path_f = os.path.join(recipe_dir, f)
+if is_recipe_or_include_file(env, full_path_f, f):
+with open(full_path_f + ".tmp", "w+") as temp_recipe:
+with open(full_path_f) as recipe:
+for line in recipe:
+func(line, temp_recipe)
+os.rename(full_path_f + ".tmp", full_path_f)
+return modify
+
 class Recipe(object):
 def __init__(self, env, new_ver, interactive, workdir, recipe_dir, 
bitbake, git):
 self.env = env
@@ -90,18 +102,12 @@ class Recipe(object):
 self.git.mv(src_dir, dest_dir)
 
 def rename(self):
-# change PR before renaming
-for f in os.listdir(self.recipe_dir):
-full_path_f = os.path.join(self.recipe_dir, f)
-if is_recipe_or_include_file(full_path_f, f):
-with open(full_path_f + ".tmp", "w+") as temp_recipe:
-with open(full_path_f) as recipe:
-for line in recipe:
-if line.startswith("PR=") or line.startswith("PR 
="):
-continue
-else:
-temp_recipe.write(line)
-os.rename(full_path_f + ".tmp", full_path_f)
+# clean PR before renaming
+@modify_recipe_files
+def _clean_pr(line, temp_recipe):
+if not (line.startswith("PR=") or line.startswith("PR =")):
+temp_recipe.write(line)
+_clean_pr(self.env, self.recipe_dir)
 
 # rename recipes (not directories)
 for path in os.listdir(self.recipe_dir):
@@ -121,8 +127,6 @@ class Recipe(object):
 # since we did some renaming, backup the current environment
 self.old_env = self.env
 
-# start formatting the commit message
-
 def create_diff_file(self, file, old_md5, new_md5):
 old_file = os.path.join(self.old_env['S'], file)
 new_file = os.path.join(self.env['S'], file)
@@ -138,20 +142,6 @@ class Recipe(object):
 f.write("old checksum = %s\n" % old_md5)
 f.write("new_checksum = %s\n" % new_md5)
 
-for f in os.listdir(self.recipe_dir):
-full_path_f = os.path.join(self.recipe_dir, f)
-if is_recipe_or_include_file(full_path_f, f):
-with open(full_path_f + ".tmp", "w+") as temp_recipe:
-with open(full_path_f) as recipe:
-for line in recipe:
-m = re.match("(.*)" + old_md5 + "(.*)", line)
-if m is not None:
-temp_recipe.write(m.group(1) + new_md5 + 
m.group(2) + "\n")
-else:
-temp_recipe.write(line)
-
-os.rename(full_path_f + ".tmp", full_path_f)
-
 def _change_recipe_checksums(self, fetch_log):
 sums = {}
 
@@ -178,27 +168,23 @@ class Recipe(object):
 if len(sums) == 0:
 raise FetchError()
 
-I(" %s: Update recipe checksums ..." % self.env['PN'])
 # checksums are usually in the main recipe but they can also be in inc
 # files... Go through the recipes/inc files until we find them
-for f in os.listdir(self.recipe_dir):
-full_path_f = os.path.join(self.recipe_dir, f)
-if is_recipe_or_include_file(full_path_f, f):
-with open(full_path_f + ".tmp", "w+") as temp_recipe:
-with open(full_path_f) as recipe:

Re: [yocto] Migration 1.8 -> 2.0 opkg-cl missing in SDK

[Paul Eggleton]

Hi Marcin,

On Wednesday 25 November 2015 17:49:19 mar.krzeminski wrote:
> After migrating from 1.8 to 2.0 opkg-cl is missing in SDK (sdk - is
> generated by bitbake meta-toolchain command).
> Manual clearly states that opkg-cl is still avaliable:
> http://www.yoctoproject.org/docs/latest/adt-manual/adt-manual.html#configuri
> ng-the-pms I can not find any instruction how to migrate PMS based on opkg
> in sdk in 2.0 release, so my question is what should I change to
> make opk running in SDK?
> I am suspecting I need to replace opkg-cl with opkg, something more is
> needed?

opkg-cl has been renamed to opkg in opkg 0.3.0 (which was included in our 2.0 
release). Probably something we ought to have noted in the 2.0 migration 
guide.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] [PSPLASH 1/2] psplash-fb.c: Fix BGR888 detection

[Burton, Ross]

On 25 November 2015 at 20:12, Aws Ismail  wrote:

> :) I have sent the same patch three years ago but no one picked it up...
>
> https://lists.yoctoproject.org/pipermail/yocto/2012-October/009846.html
>

I can only apologise.  At least it's merged now!

Ross
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Migration 1.8 -> 2.0 opkg-cl missing in SDK

[mar.krzeminski]

Hi Paul,

Thanks for clarification.

Regards,
Marcin

W dniu 25.11.2015 o 20:04, Paul Eggleton pisze:

Hi Marcin,

On Wednesday 25 November 2015 17:49:19 mar.krzeminski wrote:

After migrating from 1.8 to 2.0 opkg-cl is missing in SDK (sdk - is
generated by bitbake meta-toolchain command).
Manual clearly states that opkg-cl is still avaliable:
http://www.yoctoproject.org/docs/latest/adt-manual/adt-manual.html#configuri
ng-the-pms I can not find any instruction how to migrate PMS based on opkg
in sdk in 2.0 release, so my question is what should I change to
make opk running in SDK?
I am suspecting I need to replace opkg-cl with opkg, something more is
needed?

opkg-cl has been renamed to opkg in opkg 0.3.0 (which was included in our 2.0
release). Probably something we ought to have noted in the 2.0 migration
guide.

Cheers,
Paul



--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Copying the kernel in the poky directory

[Deepika Teriar]

Hi

I am customizing yocto for beaglebone black
I do not want my kernel to get downloaded from the git after I clean the
build directory. So i have kept the linux-kernel directory with the name
kernel-3.14.29 in the poky directory.
And in the linux-yocto_3.14.bb file i have changed SRC_URI *from:*

SRC_URI = "git://
git.yoctoproject.org/linux-yocto-3.14.git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta
"

*to :*

SRC_URI = "file://${COREBASE}/kernel-3.14.29"

On compiling core-image-minimal I am getting the following error



*ERROR: ExpansionError during parsing
/home/deepika/bbb/poky/meta/recipes-kernel/linux/linux-yocto_3.14.bb
: Failure expanding variable do_patch:
ExpansionError: Failure expanding variable SRCPV, expression was
${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher
failure: SRCREV was used yet no valid SCM was found in SRC_URI*
Can any one tell me the error?

Thanks
Deepika
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Cross-compiling python fails with new custom tuning

[Mark Hatle]

On 11/24/15 3:23 PM, Mark Hatle wrote:
> My guess is that there is a bug in the python integration where it's not
> realizing the host and target are different systems, so it's trying to run a
> target program on your host.  Your host isn't haswell, so... Illegal 
> Instruction
> -- and the system stops.

Just an FYI, I hit the same problem today.  I suspect it is the host trying to
run target software and I'm looking into it.

--Mark

> The alternative would be something is running QEMU to execute a target binary
> and QEMU doesn't have instruction support -- but that doesn't look like the 
> case
> here.
> 
> --Mark
> 
> On 11/24/15 3:06 PM, Michael Habibi wrote:
>> All,
>>
>> I added a new machine definition with tuning parameters for haswell
>> microarchitectures (basically just duplicated corei7 but tuned it for 
>> haswell).
>> This seems to work correctly, but failed when running do_install for python
>> toward the end of the build process. I am running with the Yocto 2.0 
>> framework,
>> with very minimal changes to create a new distribution and machine for our
>> custom embedded device. Note I had this distro configuration working before, 
>> and
>> the only difference is I added a new machine with this tuning.
>>
>> I believe the issue is because, as part of the do_install step for Python, it
>> attempts to run python on the local host, despite being cross-compiled. 
>> However,
>> it is tuned for a processor that my host machine doesn't run, so I get an
>> instruction exception.
>>
>> Did I do something weird? I figure python would be configured for
>> cross-compiling and therefore wouldn't try to run the target version on the
>> host, even for sanity tests. Here is the output of the failure:
>>
>> $ tail -20
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
>>
>> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
>>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
>>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
>>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
>>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
>> x86_64-diags-linux-ranlib libpython2.7.a
>> Modules/posixmodule.o: In function `posix_tmpnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
>> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
>> Modules/posixmodule.o: In function `posix_tempnam':
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
>> warning: the use of `tempnam' is dangerous, better use `mkstemp'
>> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse 
>> -mavx2
>> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
>> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python \
>> Modules/python.o \
>> -L. -lpython2.7 -lpthread -ldl  -lpthread -lutil   
>> -lm
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> python2.7 -S -m sysconfig --generate-posix-vars ;\
>> if test $? -ne 0 ; then \
>> echo "generate-posix-vars failed" ; \
>> rm -f ./pybuilddir.txt ; \
>> exit 1 ; \
>> fi
>> Illegal instruction (core dumped)
>> make: *** [sharedmods] Error 132
>> WARNING: exit code 1 from a shell command.
>> ERROR: oe_runmake failed
>> ERROR: Function failed: do_install (log file is located at
>> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258)
>>
>> Here is my tune-haswell.inc (ignore some copy/paste comment issues right now 
>> =):
>>
>> # Settings for the GCC(1) cpu-type "haswell":
>> #
>> # Intel Core i7 CPU with 64-bit extensions, MOVBE, MMX, SSE, SSE2, SSE3,·
>> # SSSE3, SSE4.1, SSE4.2, POPCNT, AVX, AVX2, AES, PCLMUL, FSGSBASE,
>> # RDRND, FMA, BMI, BMI2 and F16C instruction set support.
>> #
>> # This tune is recommended for Intel Nehalem and Silvermont (e.g. Bay Trail) 
>> CPUs
>> # (and beyond).
>> #
>> DEFAULTTUNE ?= "haswell"
>>
>> # Pull in the previous tune in to pull in PACKAGE_EXTRA_ARCHS
>> require conf/machine/include/tune-corei7.inc
>>
>> # Extra tune features
>> TUNEVALID[haswell] = "Enable haswell specific processor optimizations"
>> TUNE_CCARGS .= "${@bb.utils.contains("TUNE_FEATURES", "haswell", "
>> -march=haswell -mtune=haswell -mfpmath=sse -mavx2", "", d)}"
>>
>> # Extra tune selections
>> AVAILTUNES += "haswell"
>> 

[yocto] PRINC functionality workaround

[Belisko Marek]

Hi,

after upgrading to jethro release PRINC functionality was removed by
commit : a4d530bcf55f03258078c10a123e2717444e1060 on oe-core. PRINC
functionality was used in some of our recipes so I took part from
PRINC funtionality and put is to separate method (in bbclass) [0]

I'm trying to use it in bbappend like:
inherit veryspecial_helpers
PR := ${increase_pr(5)} but this seems to be then evaluated as :
u-boot/2015.07-${increase_pr(5)} which is wrong and fails.

Any ideas what is wrong with this approach? Maybe some brackets? Many thanks.

[0] - http://pastebin.com/zCbVaBM9

BR,

marek

-- 
as simple and primitive as possible
-
Marek Belisko - OPEN-NANDRA
Freelance Developer

Ruska Nova Ves 219 | Presov, 08005 Slovak Republic
Tel: +421 915 052 184
skype: marekwhite
twitter: #opennandra
web: http://open-nandra.com
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] [meta-intel] "Crazy" Xorg memory usage after upgrading from Daisy to Fido

[Chris Tapp]

Hi Vincent,

> On 25 Nov 2015, at 00:20, Cheah, Vincent Beng Keat 
>  wrote:
> 
> Hi Chris,
> 
> Sorry for the lateness in reply, was too busy with other things. Good to hear 
> that.
> 
> Xorg.conf is basically to force load particular intended driver (in this case 
> X11 SNA Driver). This is something that that we do not get when we bit-bake 
> for a yocto image; mainly because users from different companies/purpose may 
> want to use customized their own xorg.conf based on their needs.  You can 
> check for which X11 driver that got loaded into your Yocto system simply by 
> checking that on the /var/log/Xorg.0.log

The intel driver is being loaded when I don’t have an xorg.conf file, so I’ve 
not needed one in the past. The fbdev driver is also available, but it is 
(correctly) not selected.

What I don’t understand is:

  Option “TearFree” “true”  --- this gives normal behaviour (the same as when 
running under ‘daisy’)
  Option “TearFree” “false” --- XOrg uses all the system memory causing my app 
to be killed

I tried building the 2.99.910 version of the driver, but it fails to compile. 
I’m guessing this is down to changes in X.

> In our lab, in order to make sure that our drivers are loaded correctly, we 
> normally, build i915 as a module and add a xorg.conf files (containing some 
> of the stuff below).
> Generally, in most of the case, we will get this loaded correctly (without 
> Xorg.conf) but there do are times when we do not get the desired driver 
> loaded.
> 
> Section "Device"
>Identifier  "Card0"
>Driver  "intel"
>BusID   "0:2:0"
>Screen  0
> EndSection
> 
> ... Vincent
> 
> 
> -Original Message-
> From: Chris Tapp [mailto:opensou...@keylevel.com]
> Sent: Wednesday, November 25, 2015 7:44 AM
> To: Cheah, Vincent Beng Keat 
> Cc: Yocto Project ; Chang, Rebecca Swee Fun 
> ; Paul Eggleton 
> ; meta-in...@yoctoproject.org
> Subject: Re: [yocto] [meta-intel] "Crazy" Xorg memory usage after upgrading 
> from Daisy to Fido
> 
> Hi Vincent,
> 
> I may have made some progress. The undesirable memory usage within Xorg isn’t 
> there if I create an xorg.conf file containing:
> 
> Section “Device”
>  Identifier “Intel Video”
>  Driver “intel”
>  Option “TearFree” “true"
> EndSection
> 
> So it looks as if I need to enable “TearFree”. I didn’t need to add this for 
> the 2.99.910 version of xf86-video-intel included with ‘daisy’.
> 
> Chris
> 
>> On 23 Nov 2015, at 23:48, Chris Tapp  wrote:
>> 
>> Hi Vincent,
>> 
>> I’ve finally got back to being able to investigate this further.
>> 
>> I’ve now moved on to “jethro” and I’m seeing exactly the same behaviour. 
>> I’ve tried with kernel versions 3.14.39, 3.19.5 and 4.1.8.
>> 
>>> On 10 Jun 2015, at 03:50, Cheah, Vincent Beng Keat 
>>>  wrote:
>>> 
>>> Hi Chris,
>>> 
>>> I don’t have any idea with regard to the issue that you are getting below. 
>>> All the work that we are doing here so far is on CHV (yocto-kernel-3.19.5 
>>> standard/base branch).
>>> 
>>> From your statement below, it looks to me that you are upgrading meta-intel 
>>> from Daisy to Fido branch which are using yocto-kernel-3.14 
>>> (meta-intel/isg/valleyisland BSP). I'm not sure if you are able to 
>>> reproduce this with yocto-kernel-3.19.5 (standard/base branch) from the 
>>> meta-intel common directory. Also, comparing Daisy branch against Fido, it 
>>> seems like there are lot of changes in the user-space stacks, which I'm not 
>>> sure could cause the issue below.
>>> 
>>> 
>>> Daisy 1.6.2
>>> Kernel 3.4, 3.10, 3.14 (Supportable common base)
>>> Xorg-server 1.15
>>> Wayland/Weston 1.4.0
>>> Xf86-video-intel 2.99.910
>>> Libdrm 2.4.52
>>> MESA 9.2.5
>>> Cairo 1.12.16
>>> libVA 1.3.1 (from meta-intel)
>>> Intel-VA-driver 1.3.2 (from meta-intel)
>>> GStreamer 1.2.3
>>> GStreamer-VAAPI 0.5.8 (from meta-intel)
>>> 
>>> 
>>> Dizzy 1.7.1
>>> Kernel 3.10, 3.14, 3.17 (Supportable common base)
>>> Xorg-server 1.15.1
>>> Wayland/Weston 1.5.0
>>> Xf86-video-intel 2.99.912
>>> Libdrm 2.4.54
>>> MESA 10.1.3
>>> Cairo 1.12.16
>>> libVA 1.3.1 (from meta-intel)
>>> Intel-VA-driver 1.3.2 (from meta-intel)
>>> GStreamer 1.4.1
>>> GStreamer-VAAPI 0.5.8 (from meta-intel)
>>> 
>>> 
>>> Fido 1.8
>>> Kernel 3.14, 3.19 (supportable comon base)
>>> Xorg-server 1.16.3
>>> Wayland/weston 1.6.0
>>> Xf86-video-intel 2.99.917
>>> Libdrm 2.4.59
>>> Mesa 10.4.4
>>> Cairo 1.12.18
>>> LibVA 1.5.0 (from meta-intel)
>>> Intel-VA-driver 1.5.0 (from meta-intel)
>>> Gstreamer 1.4.5
>>> Gstreamer-vaapi 0.5.10 (from meta-intel)
>>> 
>>> 
>>> ... Vincent
>>> 
>>> -Original Message-
>>> From: Chang, 

Re: [yocto] Header files in /usr/include of rootfs

[洪翊桓_Pegatron]

Message: 2
Date: Tue, 24 Nov 2015 14:25:14 +0100
From: Anders Darander 
To: yocto@yoctoproject.org
Subject: Re: [yocto] Header files in /usr/include of rootfs
Message-ID: <20151124132514.gb2...@ad.chargestorm.se>
Content-Type: text/plain; charset=utf-8

* Andy Hung(???_Pegatron)  [151124 11:35]:
> I was trying to add some pre-built binaries and files by using a recipe.
> The recipe has something like this:

> SRC_URI = file://files ?
> do_install() {
> install ?d ${D}
> }

> What files are you installing? How does those look like?
The content of the folder has those sub-folders, sbin, root, lib, etc.
Those sub-folders contains files such as executable binaries, config 
files, kernel modules, .so files ...
All these files are actually built from the source code of a specific 
WiFi driver,
what I want to do is installing all pre-built binaries/files into 
corresponding locations in rootfs.
 
> FILES_${PN} = ?/*?

> Here you add say that anything that has been installed, but not put 
into any other package, should be added to ${PN}. Is that what you want?
Yes, all files are installed in a package.

> ALLOW_EMPTY_${PN} = ?1?

> Remove this one unless ${PN} is going to be empty.
This line is removed.

> INSANE_SKIP_${PN} = ?debug-files dev-so?

> These might be needed if you get certain QA-warnings/error without 
them.
Yes, got some QA-warnings when building so I keep this.
 
> The result was all the binaries and files installed and packages are 
> created, But I also found kernel header files are also built in under 
> /usr/include in rootfs image.
> For my understanding those header files should be used if I want to 
> develop/debug applications on the target device but it is not the case.

> Also I noticed the use of INSANE_SKIP_${PN} so I check the usage of 
> debug-files and dev-so, And it turned out there are .debug folders in the 
> WORKDIR/package folder so I thought that is why debug-files was used.

> See answer below.

> The questions are,
> Why are those kernel header files installed in rootfs? How do I prevent thisa?

> I assume that they aren't in the package you just created? Do you 
have anything in your package that might have a dependency on the kernel 
headers? Do you have anything in your IMAGE_FEATURES?
No, header files aren't in the package.
I checked the driver source code and many files include kernel headers 
like this  "#include " , is this what you meant by dependency on 
kernel headers?
IMAGE_FEATURES was not used in the recipe file.

> Why are there .debug in the package folder, it should only be in package ?dbg 
> folder, right?

> Where are the .debug directories located? You should add them to:
> FILES_${PN}-dbg += "/.debug"
I add this line and rebuild but the result seems unchanged.
the .debug folders are located in 
${WORKDIR}/package
${WORKDIR}/package-split/ - dbg

Thanks,
Andy

> Cheers,
> Anders

> --
> Anders Darander, Senior System Architect ChargeStorm AB / eStorm AB


-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[linux-yocto] framebuffer image disappear - psplash

[Ran Shalit]

Hello,


I am using psplash utility to display image on lcd.

On running ./psplash I see the image for a split of second and then it
dissapear.

Is this the behaviour that should be expected ? How can I make the
image stay on the display ?


Regards,

Ran
-- 
___
linux-yocto mailing list
linux-yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/linux-yocto

Re: [yocto] Header files in /usr/include of rootfs

[Anders Darander]

* Andy Hung(洪翊桓_Pegatron)  [151125 11:53]:

> Message: 2
> Date: Tue, 24 Nov 2015 14:25:14 +0100
> From: Anders Darander 
> To: yocto@yoctoproject.org
> Subject: Re: [yocto] Header files in /usr/include of rootfs
> Message-ID: <20151124132514.gb2...@ad.chargestorm.se>
> Content-Type: text/plain; charset=utf-8

> * Andy Hung(???_Pegatron)  [151124 11:35]:
> > I was trying to add some pre-built binaries and files by using a recipe.
> > The recipe has something like this:

> > SRC_URI = file://files ?
> > do_install() {
> > install ?d ${D}
> > }

>   > What files are you installing? How does those look like?
>   The content of the folder has those sub-folders, sbin, root, lib, etc.
>   Those sub-folders contains files such as executable binaries, config 
> files, kernel modules, .so files ...
>   All these files are actually built from the source code of a specific 
> WiFi driver,
>   what I want to do is installing all pre-built binaries/files into 
> corresponding locations in rootfs.

There's no way to build this using OE?...

Though, sure, it's should work as long as the binaries and kernel
modules actually loads and runs.

> > FILES_${PN} = ?/*?

>   > Here you add say that anything that has been installed, but not put 
> into any other package, should be added to ${PN}. Is that what you want?
>   Yes, all files are installed in a package.

> > ALLOW_EMPTY_${PN} = ?1?

>   > Remove this one unless ${PN} is going to be empty.
>   This line is removed.

> > INSANE_SKIP_${PN} = ?debug-files dev-so?

>   > These might be needed if you get certain QA-warnings/error without 
> them.
>   Yes, got some QA-warnings when building so I keep this.

> > The result was all the binaries and files installed and packages are 
> > created, But I also found kernel header files are also built in under 
> > /usr/include in rootfs image.
> > For my understanding those header files should be used if I want to 
> > develop/debug applications on the target device but it is not the case.

Could you check if any -dev packages gets installed into the image? If
so which one? Do anything like kernel-src or something similar gets
installed?

> > Also I noticed the use of INSANE_SKIP_${PN} so I check the usage of 
> > debug-files and dev-so, And it turned out there are .debug folders in the 
> > WORKDIR/package folder so I thought that is why debug-files was used.

>   > See answer below.

> > The questions are,
> > Why are those kernel header files installed in rootfs? How do I prevent 
> > thisa?

>   > I assume that they aren't in the package you just created? Do you 
> have anything in your package that might have a dependency on the kernel 
> headers? Do you have anything in your IMAGE_FEATURES?
>   No, header files aren't in the package.
>   I checked the driver source code and many files include kernel headers 
> like this  "#include " , is this what you meant by dependency on 
> kernel headers?

Are the source code packaged installed and packaged?

Can you inspect the generated package ${PN}, and see what it's depencies
are?

>   IMAGE_FEATURES was not used in the recipe file.

That would rather be in either local.conf, distro, or in your image.

> > Why are there .debug in the package folder, it should only be in package 
> > ?dbg folder, right?

>   > Where are the .debug directories located? You should add them to:
>   > FILES_${PN}-dbg += "/.debug"
>   I add this line and rebuild but the result seems unchanged.
>   the .debug folders are located in 
>   ${WORKDIR}/package
>   ${WORKDIR}/package-split/ - dbg

I don't understand. Were the .debug directories located in
packages-split/-dbg? If so, that's where we want them.

Which package do you add to your image? 
If you remove that package, do you still get the kernel headers
installed?

Cheers,
Anders

-- 
Anders Darander, Senior System Architect
ChargeStorm AB / eStorm AB
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto