[yocto] Capture download location and checksum?

[Michael Habibi]

As part of auditing, I'd like for the build to capture the download
location. I am going to work on code that generates a license manifest that
also includes the download location and checksum as part of the
'recipeinfo' file.

Is this already saved off in a variable or other construct, or do I need to
implement code to save the location used to download the file?
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Hook for layer-only actions?

[Michael Habibi]

Well I should have just used what I want to actually do as the example, and
not a QA action.

I want to create a new type of manifest that is only my layer, that will
include line items that basically has like: package name, version, download
URL, checksum, license.

I have no idea yet how to implement it, but wanted to make sure it was
feasible.

On Tue, Jul 10, 2018 at 2:27 PM Burton, Ross  wrote:

> Hypothetically a new QA action should be added at the distro level, by
> writing a class and adding it to INHERIT.
>
> (I've several new QA tests added in this way)
>
> Ross
>
> On 10 July 2018 at 19:53, Alexander Kanavin 
> wrote:
> > Yes. Implement a class and inherit it from the recipes.
> >
> > Alex
> >
> > 2018-07-10 20:50 GMT+02:00 Michael Habibi :
> >> I was wondering if there is a way I can apply a global modification to
> all
> >> recipes in a layer? For instance, we have our own layer for our changes
> that
> >> sit on top of the base Yocto/OE layers.
> >>
> >> What if, hypothetically, I wanted to insert a do_package_qa action
> globally,
> >> for everything in our layer. Is there a way to codify that, beyond
> actually
> >> adding this to every recipe in our layer?
> >>
> >> --
> >> ___
> >> yocto mailing list
> >> yocto@yoctoproject.org
> >> https://lists.yoctoproject.org/listinfo/yocto
> >>
> > --
> > ___
> > yocto mailing list
> > yocto@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Hook for layer-only actions?

[Michael Habibi]

I was wondering if there is a way I can apply a global modification to all
recipes in a layer? For instance, we have our own layer for our changes
that sit on top of the base Yocto/OE layers.

What if, hypothetically, I wanted to insert a do_package_qa action
globally, for everything in our layer. Is there a way to codify that,
beyond actually adding this to every recipe in our layer?
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Yocto SDK and license manifest

[Michael Habibi]

We use Yocto to generate a simple SDK that we use as a toolchain. However,
it appears that building the SDK vs an image does not generate the license
manifest (we have deploy/rpm, deploy/sdk, but not deploy/license).

Have we configured something wrong, or perhaps do we need to do something
additional to have the license manifest as well?
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Generating license/manifest for a specific layer?

[Michael Habibi]

So to clarify your question, basically everything we would want in the
custom manifest would be in our own layer: both bbappends and new bb
recipes. Given that, you think I could still modify it like you suggest to
get what I want?

On Wed, Jun 13, 2018 at 8:59 AM Beth Flanagan  wrote:

> Ok, I guess my question there then would be is how you'd determine
> where a recipe "lived". Like, if meta-foo has a bbappends for
> something in core, would you include it or not in your manifest?
>
> If so, then taking the original manifest and just doing some text
> manipulation in a ROOTFS_POSTPROCESS_COMMAND would be the way I'd
> probably look at tackling it.
>
> On Wed, Jun 13, 2018 at 2:42 PM, Michael Habibi 
> wrote:
> > Beth,
> >
> > This is for internal consumption. We want to be able to generate a full
> > manifest, and also one that reflects how we diverged from base Yocto
> > distribution.
> >
> > On Mon, Jun 11, 2018 at 10:55 AM Beth Flanagan 
> wrote:
> >>
> >> On Mon, Jun 11, 2018 at 2:46 PM, Michael Habibi 
> >> wrote:
> >> > Our use case is to capture the license files, manifest
> >> > (package/version),
> >> > and download information only for packages we modify/add. We use our
> own
> >> > layer to modify/add packages, everything coming from standard Yocto
> >> > layers
> >> > are untouched.
> >> >
> >> > Is there a way to generate this information on a layer-by-layer basis,
> >> > instead of a full manifest that includes all standard, unmodified
> >> > packages?
> >>
> >> The easy (cheating) way, would be to modify the tmp/deploy/licenses
> >> artifact post build (I do it to remove -native- and -cross- from
> >> things I distribute as I'm not actually distributing them) or put in a
> >> post do_rootfs function that does it.
> >>
> >> I guess my question would be (and this is less a technical question
> >> and more of a legal one) is why would you want to only include a
> >> manifest for only part of what you're distributing (or am I
> >> misunderstanding what you're trying to do here?)
> >>
> >> -b
> >> >
> >> > --
> >> > ___
> >> > yocto mailing list
> >> > yocto@yoctoproject.org
> >> > https://lists.yoctoproject.org/listinfo/yocto
> >> >
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Generating license/manifest for a specific layer?

[Michael Habibi]

Beth,

This is for internal consumption. We want to be able to generate a full
manifest, and also one that reflects how we diverged from base Yocto
distribution.

On Mon, Jun 11, 2018 at 10:55 AM Beth Flanagan  wrote:

> On Mon, Jun 11, 2018 at 2:46 PM, Michael Habibi 
> wrote:
> > Our use case is to capture the license files, manifest (package/version),
> > and download information only for packages we modify/add. We use our own
> > layer to modify/add packages, everything coming from standard Yocto
> layers
> > are untouched.
> >
> > Is there a way to generate this information on a layer-by-layer basis,
> > instead of a full manifest that includes all standard, unmodified
> packages?
>
> The easy (cheating) way, would be to modify the tmp/deploy/licenses
> artifact post build (I do it to remove -native- and -cross- from
> things I distribute as I'm not actually distributing them) or put in a
> post do_rootfs function that does it.
>
> I guess my question would be (and this is less a technical question
> and more of a legal one) is why would you want to only include a
> manifest for only part of what you're distributing (or am I
> misunderstanding what you're trying to do here?)
>
> -b
> >
> > --
> > ___
> > yocto mailing list
> > yocto@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
> >
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Generating license/manifest for a specific layer?

[Michael Habibi]

Our use case is to capture the license files, manifest (package/version),
and download information only for packages we modify/add. We use our own
layer to modify/add packages, everything coming from standard Yocto layers
are untouched.

Is there a way to generate this information on a layer-by-layer basis,
instead of a full manifest that includes all standard, unmodified packages?
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Patching only the target version of a package?

[Michael Habibi]

Thanks, that seems to be pretty much precisely what I need. Where can I
learn more about the class construct, e.g. class-target and native? I saw
some references to it in the docs, but not an actual explanation of how
that all works and is put together. Any ideas?

On Thu, Feb 1, 2018 at 9:34 AM, Burton, Ross <ross.bur...@intel.com> wrote:

> On 1 February 2018 at 15:03, Michael Habibi <mikehab...@gmail.com> wrote:
>
>> So let's say my whole bbappend file is only necessary for the target
>> version. Can I rename the whole thing package-target.bbappend (add the
>> -target suffix) and it will only append it to the recipe when building the
>> target? Is "target" the actual suffix or is it a placeholder for something
>> more specific? I have only seen -native, -cross, and some other ones that
>> don't seem to apply to this case.
>>
>>
> That won't work, as native and cross suffixes in recipes are for humans,
> not bitbake.
>
> Ross
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Patching only the target version of a package?

[Michael Habibi]

So let's say my whole bbappend file is only necessary for the target
version. Can I rename the whole thing package-target.bbappend (add the
-target suffix) and it will only append it to the recipe when building the
target? Is "target" the actual suffix or is it a placeholder for something
more specific? I have only seen -native, -cross, and some other ones that
don't seem to apply to this case.

On Thu, Feb 1, 2018 at 8:44 AM, Burton, Ross <ross.bur...@intel.com> wrote:

> On 1 February 2018 at 14:29, Michael Habibi <mikehab...@gmail.com> wrote:
>
>> I'm sure this is really simple but I haven't quite wrapped my head around
>> it.
>>
>> We have a bbappend file that supplies a set of patches. It currently has
>> the unintended side-effect of patching both the native version used during
>> the Yocto build process, and the eventual target version. How do I modify
>> the recipe such that it only acts upon the target version?
>>
>> Would it be something like recipe-cross.bbappend instead of
>> recipe.bbappend? Or something similar?
>>
>
>
> SRC_URI_append_class-target = " file://..."
>
> Ross
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Patching only the target version of a package?

[Michael Habibi]

I'm sure this is really simple but I haven't quite wrapped my head around
it.

We have a bbappend file that supplies a set of patches. It currently has
the unintended side-effect of patching both the native version used during
the Yocto build process, and the eventual target version. How do I modify
the recipe such that it only acts upon the target version?

Would it be something like recipe-cross.bbappend instead of
recipe.bbappend? Or something similar?
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Using site.conf

[Michael Habibi]

Ok thanks! I think I mistakenly thought that I needed to have it be called
site.conf.sample, and it will copy the sample into the .conf file in the
build/conf area.

On Tue, Oct 31, 2017 at 9:24 AM, Burton, Ross <ross.bur...@intel.com> wrote:

> On 31 October 2017 at 14:15, Michael Habibi <mikehab...@gmail.com> wrote:
>
>> Is there an example of this somewhere? I haven't had any luck in
>> documentation or example code to see how to make the site.conf be used.
>>
>>
> site.conf is loaded if it is present in a conf/ directory anywhere.  I
> have a meta-ross which contains conf/site.conf, I add that layer to every
> build tree and bitbake will read the site.conf.  There's not a lot more to
> it really.
>
> Ross
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Using site.conf

[Michael Habibi]

Is there an example of this somewhere? I haven't had any luck in
documentation or example code to see how to make the site.conf be used.

On Mon, Oct 30, 2017 at 6:16 PM, Khem Raj <raj.k...@gmail.com> wrote:

> On Fri, Oct 27, 2017 at 11:59 AM, Michael Habibi <mikehab...@gmail.com>
> wrote:
> > I'm a bit confused about the usage of site.conf. I see a sample in the
> poky
> > distro area, but I'm not sure how this actually gets copied into a
> site.conf
> > and actually used by deployed versions of our distribution.
> >
> > Is there some extra step I need to take to ensure site.conf.sample from
> my
> > own distributions conf directory gets copied and actually used by our
> > engineers' environments?
>
> yes, you need to do that as part of workspace setup.
>
> >
> > --
> > ___
> > yocto mailing list
> > yocto@yoctoproject.org
> > https://lists.yoctoproject.org/listinfo/yocto
> >
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Using site.conf

[Michael Habibi]

I'm a bit confused about the usage of site.conf. I see a sample in the poky
distro area, but I'm not sure how this actually gets copied into a
site.conf and actually used by deployed versions of our distribution.

Is there some extra step I need to take to ensure site.conf.sample from my
own distributions conf directory gets copied and actually used by our
engineers' environments?
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Git/yocto newbie question re: submodules and multiple layers

[Michael Habibi]

Hi, I am well-versed with Git but I haven't really ever had to use
submodules. I am trying to create a git repository on our server that
will host our yocto distribution. Our distribution will include the
yocto repo, plus meta-openembedded repo, plus maybe some other layers.

The difficulty I have is thus:

my_git_repo/ <--- what my platform developers will 'git clone'
my_git_repo/yocto_git/... <-- yocto framework
my_git_repo/yocto_git/meta-openembedded <-- another git repo nested
inside yocto repo
my_git_repo/yocto_git/meta-intel <-- another example layer.

I would like for developers to be able to 'git clone' a single repo,
but pull down all the necessary layers from various git repos.
Basically it will comprise all repos, including yocto, OE layers,
intel layers, etc.

I believe the only way I can nest submodules inside pre-existing repos
is for me to create a local clone of yocto git and add the submodules
to that clone. That means the master git repo (my_git_repo in example
above) will point to *our* clone of yocto git and not yocto project's
git repo. In our local clone of yocto git, I will add a submodule for
each layer I want to add.

Does that make sense? Is this the right approach or is there a smarter way?
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Building applications and distributions separately, but integrate into final image

[Michael Habibi]

All,

I am looking at Yocto as a replacement for our embedded distribution.
Currently we build everything as a distribution. This includes
building the Linux kernel, open source packages, as well as our own IP
and applications.

As you can imagine, a vast majority of our build is taken up by
repeatedly building distribution components and not our actual IP,
which is a majority of what changes day to day.

In moving to Yocto, my first plan was to create all the necessary
layers and separate out what changes from what doesn't (in fact much
of what doesn't change is already included in Yocto or the many
freely-available layers online).

However, this would still require application developers to have to
learn and use Yocto to build the entire distribution. Ideally, I'd
like to separate the applications from the main build, so that
application developers have to learn less about Yocto, and more
importantly, save on build time when they make their changes.

I am exploring exactly how I can use the available SDK options as a
solution. If I create an image with the base distribution and required
applications, I'm wondering if I can create a separate tree for the
application developers. They would then build that entire tree with
the yocto SDK. I haven't quite figured out yet how I can then
integrate their applications into the Yocto image, and prepare it for
deployment. I'm also unclear how we can define new dependencies if the
base distribution is pre-built. For example, if we introduce a new
dependency (our custom application now requires zlib, for example), it
doesn't seem for there to be a way to then ensure zlib is built into
the final image with the SDK option.

Unfortunately, we can't build the distribution and then install the
applications directly onto the target. Based on how our codebase is
currently designed, it requires a fully-complete image to be installed
on the target as a whole, and not piecemeal. This is why I need to
integrate on the host/build environment first before we create the
deployable image.

Can anyone share any workflows that they use that may fit with what we
need? In the mean time I am continuing to play with Yocto and the SDK
to see what else I can come up with.

Thanks all for your time.
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Trying to run toaster off master, running into issues w/ django

[Michael Habibi]

Excellent. Thanks! I knew using latest was going to burn me eventually :)

On Wed, Aug 24, 2016 at 5:51 AM, Ed Bartosh <ed.bart...@linux.intel.com> wrote:
> On Tue, Aug 23, 2016 at 04:49:42PM -0500, Michael Habibi wrote:
>> I am running the master branch and trying to run toaster to play
>> around with the web interface (already successfully completed a build
>> w/o toaster setup).
>>
>> I followed the instructions in the documentation. I created a
>> virtualenv, sourced/activated my virtualenv, and while under venv I
>> did a pip install. It believes Django is installed, but I cannot run
>> toaster. See below:
>>
>> (venv)habibim@bvm-poc8:/projects/yocto-git/build$ pip install -r
>> /projects/yocto-git/bitbake/toaster-requirements.txt
>> Requirement already satisfied (use --upgrade to upgrade):
>> Django>1.8,<1.9 in /home/habibim/venv/lib/python2.7/site-packages
>> (from -r /projects/yocto-git/bitbake/toaster-requirements.txt (line
>> 1))
>> Requirement already satisfied (use --upgrade to upgrade):
>> beautifulsoup4>=4.4.0 in
>> /home/habibim/venv/lib/python2.7/site-packages (from -r
>> /projects/yocto-git/bitbake/toaster-requirements.txt (line 2))
>> Requirement already satisfied (use --upgrade to upgrade): pytz in
>> /home/habibim/venv/lib/python2.7/site-packages (from -r
>> /projects/yocto-git/bitbake/toaster-requirements.txt (line 3))
>> Cleaning up...
>> (venv)habibim@bvm-poc8:/projects/yocto-git/build$ python --version
>> Python 2.7.6
>>
>> ...
>>
>>
>> (venv)habibim@bvm-poc8:/projects/yocto-git$ source oe-init-build-env
>>
>> ### Shell environment set up for builds. ###
>>
>> You can now run 'bitbake '
>>
>> Common targets are:
>> core-image-minimal
>> core-image-sato
>> meta-toolchain
>> meta-ide-support
>>
>> You can also run generated qemu images with a command like 'runqemu qemux86'
>> (venv)habibim@bvm-poc8:/projects/yocto-git/build$ source 
>> ../bitbake/bin/toaster
>> Traceback (most recent call last):
>>   File "", line 1, in 
>> ImportError: No module named 'django'
>> This program needs Django>1.8,<1.9
>> Please install with pip install -r
>> /projects/yocto-git/bitbake/toaster-requirements.txt
>>
>> Am I going insane? I did some google-fu and people recommended
>> cleaning up manage.py, but those changes did not seem to affect this
>> outcome.
>
> Looks like python3 issue to me. You should install requirements with
> pip3 for python3 to be able to import them.
>
> You probably used old setup instructions. Please, read the latest manual here:
> http://www.yoctoproject.org/docs/latest/toaster-manual/toaster-manual.html#toaster-manual-setup-and-use
>
> Regards,
> Ed
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Build toolchain only (crosstools substitute?)

[Michael Habibi]

If I needed to modify glibc or gcc build options (hypothetically speaking,
to turn on profiling or debug), I assume I can do that with bbappends and
the like?

Thanks Paul

On Mon, May 2, 2016 at 11:51 PM, Paul Eggleton <
paul.eggle...@linux.intel.com> wrote:

> Hi Michael,
>
> On Mon, 02 May 2016 16:38:16 Michael Habibi wrote:
> > I have been a few months out of researching Yocto, so I'm having trouble
> > caching it all back in (a lot to absorb!). Excuse me if this is answered
> > clearly in the documentation (I glanced around and I know you can build
> > relocatable toolchains, etc, but I suppose I'm asking more of a
> > philosophical question).
> >
> > I have long-term plans to change our distribution at work over from a
> > custom distribution to one built by Yocto - but this is a few months to a
> > year out.
> >
> > In the mean time, we need a new toolchain simply to update our glibc and
> > gcc packages. I understand that yocto 2.1 can build gcc 5.3 w/ glibc 2.23
> > support. I was wondering if this would be considered an appropriate
> > exercise to use the yocto framework to build solely a toolchain to be
> used
> > with a custom distribution (understanding that the custom distribution
> > would have to be modified to source and use the new toolchain).
> >
> > We typically use crosstools to generate the toolchain, but it is a bit
> > outdated and only builds gcc 5.2 (which has some bugs that were fixed in
> > 5.3). Essentially I want to use Yocto recipes as a substitute for
> > crosstools-ng.
>
> If you build (or download) our SDK that should give you what you need:
>
>
> http://www.yoctoproject.org/docs/2.1/sdk-manual/sdk-manual.html#sdk-building-an-sdk-installer
>
> If it's just the toolchain you need you can ignore the bits about
> the extensible SDK - the standard SDK is what you want. There's
> some more explanatory stuff towards the top of that manual as well.
>
> Cheers,
> Paul
>
> --
>
> Paul Eggleton
> Intel Open Source Technology Centre
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Build toolchain only (crosstools substitute?)

[Michael Habibi]

I have been a few months out of researching Yocto, so I'm having trouble
caching it all back in (a lot to absorb!). Excuse me if this is answered
clearly in the documentation (I glanced around and I know you can build
relocatable toolchains, etc, but I suppose I'm asking more of a
philosophical question).

I have long-term plans to change our distribution at work over from a
custom distribution to one built by Yocto - but this is a few months to a
year out.

In the mean time, we need a new toolchain simply to update our glibc and
gcc packages. I understand that yocto 2.1 can build gcc 5.3 w/ glibc 2.23
support. I was wondering if this would be considered an appropriate
exercise to use the yocto framework to build solely a toolchain to be used
with a custom distribution (understanding that the custom distribution
would have to be modified to source and use the new toolchain).

We typically use crosstools to generate the toolchain, but it is a bit
outdated and only builds gcc 5.2 (which has some bugs that were fixed in
5.3). Essentially I want to use Yocto recipes as a substitute for
crosstools-ng.

Thoughts on this? Terrible idea? Decent idea?

Thanks,
Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Where root password is defined by default

[Michael Habibi]

I see an example of setting the root password in the extrausers class
documentation -

http://www.yoctoproject.org/docs/2.0/ref-manual/ref-manual.html#ref-classes-extrausers

This sets the password to 1876*18:

 inherit extrausers
 EXTRA_USERS_PARAMS = "\
 usermod -P 1876*18 root; \
 "


On Tue, Jan 12, 2016 at 5:58 AM, Vivek Per  wrote:

> Hi,
>  Where root password is defined by default. How can we change it
>
> Thanks and regards
>   Vivek
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Power off IC over Yocto

[Michael Habibi]

Simon, that's more of a kernel development question and probably doesn't
fit in the Yocto project mailing list. You may try the linux kernel
development mailing lists.

However, I will give you some feedback. I think you are modifying code too
deep in the kernel. By the time you get to that function, the system should
not be executing more tasks. If it does, it will likely get terminated
pretty quickly afterward, regardless if it finished (no guarantee your
process will run to completion). It's also not great practice to actually
modify the kernel itself, because then it will only work with your one
machine/setup (unless that is acceptable). Kernel modules loaded in may be
better practice here.

I don't understand your question about machine_halt, but I will assume you
are asking how important the lines of code are between your usermode helper
call and machine_halt. The answer is very. All of those lines must execute
successfully to have clean shutdown.

On Wed, Dec 16, 2015 at 6:37 AM, simo  wrote:

> Hi,
>
> right now we solved executing an external tool we wrote in C to send the
> I2C command to the external expansion board.
>
> I had to modify the kernel_halt() function as follows:
>
> void kernel_halt(void)
> {
>
> /* added by simo */
>
> /* external environment declaration */
> char i2c_shutdown_cmd[] = "/home/root/i2c-shutdown";
> char **argv;
> static char *envp[] =
> {
> "HOME=/",
> "PATH=/home/root:/sbin:/usr/sbin:/bin:/usr/bin",
> NULL
> };
>
> argv = argv_split(GFP_KERNEL, i2c_shutdown_cmd, NULL);
>
> // WORKS HERE
> pr_emerg("Turning off.\n");
> call_usermodehelper(argv[0], argv, envp, UMH_WAIT_EXEC);
>
> /* no more executed */
> kernel_shutdown_prepare(SYSTEM_HALT);
>
> migrate_to_reboot_cpu();
> syscore_shutdown();
> //pr_emerg("System halted\n");
> kmsg_dump(KMSG_DUMP_HALT);
>
> /* commented while(1); in machine_halt for arm */
> machine_halt();
> }
> EXPORT_SYMBOL_GPL(kernel_halt);
>
> I cannot execute call_usermodehelper after kernel_shutdown_prepare()
> because this function disables user space access.
>
> So I have a question: how much is important to execute the code until
> machine_halt() ?
>
> I am using a kernel version 3.18.18.
>
> Regards,
> Simon
>
>
> El mar, 15-12-2015 a las 19:46 +0100, Christian Ege escribió:
> >
> > Am 15.12.2015 7:45 nachm. schrieb "Christian Ege" :
> > >
> > > Hi,
> > >
> > > Am 15.12.2015 6:45 nachm. schrieb "simo"
> > :
> > > >
> > > > I am using a Gumtix Overo board and we create and expansion board
> > that
> > > > power Overo and other devices by external power supply IC.
> > > >
> > > > Our expansion board with power supply are connected by I2C bus 2
> > in
> > > > overo (not the same i2C that TW4030 or tps6595 is connected).
> > > >
> > > > Using the Linux  command shutdown we would like to power off the
> > hole
> > > > system.
> > > > We would like to close the linux and last thing send a command
> > over I2C
> > > > to our power supply IC in order it disconnect the physical power.
> > > >
> > > > Our question is:
> > > >
> > > > Where we have to implement that function?
> > > > Is there any example of it?
> > > I guess you can use
> > > http://lxr.free-electrons.com/source/kernel/reboot.c
> > >
> > register_reboot_notifier is your friend
> > > You have to create some kernel module doing this or write a kernel
> > patch.
> > >
> > > Regards,
> > > Christian
> > >
> > > > --
> > > > ___
> > > > yocto mailing list
> > > > yocto@yoctoproject.org
> > > > https://lists.yoctoproject.org/listinfo/yocto
> >
> >
>
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] populate_sdk fails almost immediately

[Michael Habibi]

I'm not sure if this was the cause, but I did two things and it fixed:

1) I changed SDK_VENDOR from "-diags-sdk" to "-diagssdk"
2) I deleted /tmp

Perhaps having two dashes in SDK_VENDOR was screwing up a parse statement
somewhere.

On Wed, Dec 16, 2015 at 2:55 PM, Michael Habibi <mikehab...@gmail.com>
wrote:

> I decided to built the SDK to play around with the output so I can
> understand how it all goes together. According to the documentation, I
> simply do:
>
> bitbake  -c populate_sdk
>
> It's worth noting I am running a custom image and distro and not a
> poky/yocto image (I do inherit core-image, however). I don't spot any
> critical differences between core-image-minimal and my custom image. My
> distro conf includes poky.conf to pull in most of the configuration.
> However, the first thing I get is this:
>
> DEBUG: Executing python function sysroot_cleansstate
> DEBUG: Python function sysroot_cleansstate finished
> DEBUG: SITE files ['endian-little', 'common-linux', 'common-glibc',
> 'bit-64', 'x86_64-linux', 'common']
> DEBUG: Executing shell function autotools_preconfigure
> DEBUG: Shell function autotools_preconfigure finished
> DEBUG: Executing python function autotools_copy_aclocals
> DEBUG: SITE files ['endian-little', 'common-linux', 'common-glibc',
> 'bit-64', 'x86_64-linux', 'common']
> DEBUG: Python function autotools_copy_aclocals finished
> DEBUG: Executing shell function do_configure
> configure.ac:34: error: Please use exactly Autoconf 2.64 instead of 2.69.
> config/override.m4:12: _GCC_AUTOCONF_VERSION_CHECK is expanded from...
> configure.ac:34: the top level
>
> for the binutils-crosssdk do_configure task. Am I doing something insane?
> I figure since my image build is working correctly, it should be using the
> right versions of autoconf, etc.
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] populate_sdk fails almost immediately

[Michael Habibi]

I decided to built the SDK to play around with the output so I can
understand how it all goes together. According to the documentation, I
simply do:

bitbake  -c populate_sdk

It's worth noting I am running a custom image and distro and not a
poky/yocto image (I do inherit core-image, however). I don't spot any
critical differences between core-image-minimal and my custom image. My
distro conf includes poky.conf to pull in most of the configuration.
However, the first thing I get is this:

DEBUG: Executing python function sysroot_cleansstate
DEBUG: Python function sysroot_cleansstate finished
DEBUG: SITE files ['endian-little', 'common-linux', 'common-glibc',
'bit-64', 'x86_64-linux', 'common']
DEBUG: Executing shell function autotools_preconfigure
DEBUG: Shell function autotools_preconfigure finished
DEBUG: Executing python function autotools_copy_aclocals
DEBUG: SITE files ['endian-little', 'common-linux', 'common-glibc',
'bit-64', 'x86_64-linux', 'common']
DEBUG: Python function autotools_copy_aclocals finished
DEBUG: Executing shell function do_configure
configure.ac:34: error: Please use exactly Autoconf 2.64 instead of 2.69.
config/override.m4:12: _GCC_AUTOCONF_VERSION_CHECK is expanded from...
configure.ac:34: the top level

for the binutils-crosssdk do_configure task. Am I doing something insane? I
figure since my image build is working correctly, it should be using the
right versions of autoconf, etc.
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Custom license in file header

[Michael Habibi]

I am trying to pull in the stream benchmark utility into my custom layer.
However this appears to have a custom license. It's in the header of
stream.c, but it's also copied here:

https://www.cs.virginia.edu/stream/FTP/Code/LICENSE.txt

For simplicity, I am simply using stream.c and providing beginline/endline
parameters to get the proper section of the C header, e.g.:

LIC_FILES_CHKSUM = "file://stream.c;beginline=13;endline=41"


That allows me to properly set LIC_FILES_CHKSUM - but what do I set for
LICENSE? It's a completely non-standard license. Do I have to copy this
license somewhere as a common license, despite it only being used for this
one utility? A bit unclear about these details.

Thanks
Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Understanding build work directories (machine vs tuning)

[Michael Habibi]

I have created a new machine, and that new machine has a tuning specific to
our architecture. For example, machine=mymachine and mymachine's default
tuning is myarch.

I've noticed that for the target, we actually get two different work
directories: one based on my machine, e.g. mymachine-distro-linux, and one
based on my tuning, e.g. myarch-distro-linux. I haven't quite figured out
the difference between what gets compiled into myarch vs mymachine, as I
get some of both in my rootfs.

I'm sure I'm missing something obvious, but thought I'd ask.

Thanks all,
Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Best practice for building a large amount of source files

[Michael Habibi]

If I have some source I want to bring into my layer, for example:

Recipe dir:
meta-mylayer/recipes-management/myapplication/myapplication_1.0/

What is the best option for adding all of these and compiling? As far as I
know, you have to do one of the following:

1) Use a fetch that fetches a tarball with all your source, or a repo with
all of your source
2) Use a million-line long SRC_URI assignment to include every file that
your application uses.

Is there a third (or fourth) option that I may have missed?

Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Best practice for building a large amount of source files

[Michael Habibi]

Ross, I will likely try a different workflow if we enter production
environment with yocto. Right now I am just doing yocto integration testing
myself, and want to simplify things as much as possible for myself :)

On Thu, Dec 10, 2015 at 3:54 PM, Burton, Ross <ross.bur...@intel.com> wrote:

>
> On 10 December 2015 at 21:16, Michael Habibi <mikehab...@gmail.com> wrote:
>
>> What is the best option for adding all of these and compiling? As far as
>> I know, you have to do one of the following:
>>
>> 1) Use a fetch that fetches a tarball with all your source, or a repo
>> with all of your source
>> 2) Use a million-line long SRC_URI assignment to include every file that
>> your application uses.
>>
>> Is there a third (or fourth) option that I may have missed?
>>
>
> (1) is definitely my recommendation, separate the application development
> from the recipe to build the application.
>
> Ross
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] BeagleBone...bitbake -v core-image-sato stops

[Michael Habibi]

You may be able to look under ${WORKDIR}/temp/log.do_compile to see what
the logs are saying it got stuck doing.

On Sun, Dec 6, 2015 at 4:16 AM,  wrote:

> Hello to all,
>
> this is my first question ;-) I'm doing my first steps with a beaglebone
> black and yocto. I've bought the beaglebone book "Using Yocto Project with
> Beaglebone Black" and started the bitbake with "bitbake -v
> core-image-sato". After a while the bitbake-process reached a package
> called attr-native-2.4.47-r0 and it was not able to move on. Bitmaker did
> not complain about anything, but I've started the process yesterday in the
> morning and until now there was no progress.
>
> I've downloaded the source with:
>
> git clone -b daisy git://git.yoctoproject.org/poky.git
>
> My Distro is:
>
> gentoo
>
> I've started the bitbake with:
>
> bitbake -v core-image-sato
>
>
> Now I'm wondering, if I did something wrong .
>
> Greetings,
>
> kaffeesurrogat
>
>
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Best practice for files copied directly into sysroot?

[Michael Habibi]

Yes you're right thanks for correcting - I meant rootfs!

On Sat, Dec 5, 2015 at 3:23 PM, Paul Eggleton <paul.eggle...@linux.intel.com
> wrote:

> Hi Michael,
>
> On Tue, 01 Dec 2015 09:26:15 Michael Habibi wrote:
> > I am working on potentially migrating our distribution to Yocto (will be
> a
> > long, long process). We have an area of our filesystem where any file
> > placed will go directly into the rootfs of the target. This works well
> with
> > our product, as we have certain binaries and scripts that need to be
> copied
> > directly, and there's no other real step involved (no compilation, etc).
> Is
> > there a best practice or existing class that handles this? I saw mention
> of
> > a binary/ directory but I wasn't sure if that was for this or something
> > else.
>
> I see you got your answer - but I wanted to point out when you wrote
> "sysroot"
> you meant "rootfs" - these two aren't the same thing. The sysroot is the
> area
> used to share files between recipes (e.g. libraries and headers); whilst
> it can
> look a bit similar to what ends up in the rootfs it's actually completely
> separate.
>
> Cheers,
> Paul
>
> --
>
> Paul Eggleton
> Intel Open Source Technology Centre
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Best practice for files copied directly into sysroot?

[Michael Habibi]

This is perfect, thanks Bryan.

On Fri, Dec 4, 2015 at 8:03 AM, Bryan Evenson <beven...@melinkcorp.com>
wrote:

> Michael,
>
> > -Original Message-
> > From: Michael Habibi [mailto:mikehab...@gmail.com]
> > Sent: Thursday, December 03, 2015 5:54 PM
> > To: Bryan Evenson <beven...@melinkcorp.com>
> > Cc: yocto@yoctoproject.org
> > Subject: Re: [yocto] Best practice for files copied directly into
> sysroot?
> >
> > Bryan,
> >
> > I looked through the documentation and I'm a bit unclear on how to use
> > bin_package, and haven't been able to find a good example. Where do I
> > actually place my rootfs files/binaries to get copied to my image rootfs?
> > Under ${THISDIR}/${PN} like other recipe artifacts? How does it copy
> them to
> > ${S}? Or is this something I need to do?
> >
>
> I just modified one of my recipes yesterday to use bin_package yesterday
> and found out a few nuances that make it easier to use.  Proper setup of
> the files to copy is key.  The manual states " The bin_package class works
> well when the files extracted into ${S} are already laid out in the way
> they should be laid out on the target."  By default, any files listed as
> "file://" in SRC_URI are copied to ${WORKDIR}, and if they are a compressed
> archive (.tar.gz, .zip) they are uncompressed under ${WORKDIR}.  If you
> create a tarball with the files in the locations (and permissions!) that
> you want on the final image, then the bin_package class will do the rest of
> the work.  However, you need the files in the tarball to be one directory
> level down so that they are installed on the image properly.
>
> Let's assume you have a recipe under meta-mylayer/recipes-mine/my-app/
> called my-app_1.0.0.bb.  You have an executable called my-app that you
> want to install in /usr/bin/, a configuration file called my-app.conf that
> you want to install at /usr/share/my-app/ and documentation called
> my-app.pdf you want to install at /usr/share/info/.  You want all this to
> be under a directory so that when the tarball gets unpacked under
> ${WORKDIR} the files and directories you want on the final image are under
> ${S}.  ${S} defaults to ${WORKDIR}/${BP}, which in this case would be
> ${WORKDIR}/my-app-1.0.0.  So if you created the following directory
> structure:
>
> my-app-1.0.0/
> usr/
> bin/
> my-app
> info/
> my-app.pdf
> my-app/
> my-app.conf
>
> Again, remember to set the directory and file permissions as you want on
> the final image.  Then create a tarball from the directory you laid out
> called my-app-1.0.0.tar.gz and place it under
> meta-mylayer/recipes-mine/my-app/files/.  Then create your recipe as:
>
> SUMMARY = "summary here"
> SECTION = "proper section name here"
> LICENSE = "license name here"
> LIC_FILES_CHKSUM = "license path and MD5sum here"
>
> SRC_URI = "file://${BP}.tar.gz"
> inherit bin_package
>
> The system will then package your files and install them onto the image
> where they belong.  These are the only lines you need in your recipe other
> than the standard header variables.  In the future, if you have a new
> version then just recreate the tarball with the updated files and rename
> the tarball to match the new version.  Although I am having issues with the
> license; I haven't figured out where to put it and what to set
> LIC_FILES_CHKSUM to in the recipe so that bitbake can find the license.  If
> anyone has a good answer for that, let me know.
>
> Regards,
> Bryan
>
> > Thanks,
> > Michael
> >
> > On Tue, Dec 1, 2015 at 2:03 PM, Michael Habibi <mikehab...@gmail.com
> > <mailto:mikehab...@gmail.com> > wrote:
> >
> >
> >   Thanks Bryan!
> >
> >   On Tue, Dec 1, 2015 at 1:50 PM, Bryan Evenson
> > <beven...@melinkcorp.com <mailto:beven...@melinkcorp.com> > wrote:
> >
> >
> >   Michael,
> >
> >
> >   > -Original Message-
> >   > From: yocto-boun...@yoctoproject.org <mailto:yocto-
> > boun...@yoctoproject.org>  [mailto:yocto- <mailto:yocto->
> >   > boun...@yoctoproject.org
> > <mailto:boun...@yoctoproject.org> ] On Behalf Of Michael Habibi
> >   > Sent: Tuesday, December 01, 2015 10:26 AM
> >   > To: yocto@yoctoproject.org
> > <mailto:yocto@yoctoproject.org>
> >   > Subject: [yocto] Best practice for files copied directly
> into
> > sysroot?
> > 

[yocto] Wilcard/recursing for SRC_URI file fetcher?

[Michael Habibi]

As far as I can tell, there is no way to point to a directory or match
based on file type, for example:

SRC_URI = "file://*.c \
   file://*.h"
or

SRC_URI = "file://directory/;recurse=true" # include all files under
'directory'

Instead, files must be individually selected with the SRC_URI command. I
kind of assume this is intentional, to explicitly name files to prevent
error-prone operations. However, in my environment it may actually be
beneficial for something like the above.

Has it been considered already? Is it "taboo" or "off-limits", or will the
project welcome the enhancement if it were done?

Michael

P.S. This is of course assuming the feature isn't already there, but I
couldn't find this behavior in the reference docs. I could be mistaken!
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Pure initramfs image

[Michael Habibi]

All,

I have interest in creating an image that runs purely in an initramfs
space. It seems that overall this shouldn't be too difficult - I can set my
IMAGE_FSTYPE to cpio.gz, and as long as it's a reasonable size, it
shouldn't be an issue to point to it as an initrd. The only quirk is that
kernel modules are split out from the rootfs deployment (two separate
archives). Is there a way to combine the modules directly into the rootfs
archive?

Thanks,
Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Best practice for files copied directly into sysroot?

[Michael Habibi]

Bryan,

I looked through the documentation and I'm a bit unclear on how to use
bin_package, and haven't been able to find a good example. Where do I
actually place my rootfs files/binaries to get copied to my image rootfs?
Under ${THISDIR}/${PN} like other recipe artifacts? How does it copy them
to ${S}? Or is this something I need to do?

Thanks,
Michael

On Tue, Dec 1, 2015 at 2:03 PM, Michael Habibi <mikehab...@gmail.com> wrote:

> Thanks Bryan!
>
> On Tue, Dec 1, 2015 at 1:50 PM, Bryan Evenson <beven...@melinkcorp.com>
> wrote:
>
>> Michael,
>>
>> > -Original Message-
>> > From: yocto-boun...@yoctoproject.org [mailto:yocto-
>> > boun...@yoctoproject.org] On Behalf Of Michael Habibi
>> > Sent: Tuesday, December 01, 2015 10:26 AM
>> > To: yocto@yoctoproject.org
>> > Subject: [yocto] Best practice for files copied directly into sysroot?
>> >
>> > I am working on potentially migrating our distribution to Yocto (will
>> be a long,
>> > long process). We have an area of our filesystem where any file placed
>> will go
>> > directly into the rootfs of the target. This works well with our
>> product, as we
>> > have certain binaries and scripts that need to be copied directly, and
>> there's
>> > no other real step involved (no compilation, etc). Is there a best
>> practice or
>> > existing class that handles this? I saw mention of a binary/ directory
>> but I
>> > wasn't sure if that was for this or something else.
>> >
>> > Thanks,
>> > Michael
>>
>> I'm glad I decided to answer your question because I think I found an
>> easier way to do it than I have been doing this in the past.  If you want
>> to be able to update the files on your system at a later date, you will
>> want to make your own recipe which installs the binaries and scripts onto
>> the rootfs.  In general you create a recipe which does not have
>> do_configure or do_compile tasks.  There is a class that does this already
>> called "bin_package" that does this for you.  See the "Packaging Externally
>> Produced Binaries" section of the manual here:
>> http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html#new-recipe-testing-examples.
>>  So if you include the line "inherit bin_package" and follow the
>> instructions in that section of the manual, it should install all the files
>> to the rootfs that are included in the recipe.
>>
>> Regards,
>> Bryan
>>
>>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Pure initramfs image

[Michael Habibi]

Oh well that's easy. I figured it was already added because it was in the
deployed directory, but I guess the tasks behave differently.

On Thu, Dec 3, 2015 at 10:32 AM, Gary Thomas <g...@mlbassoc.com> wrote:

> On 2015-12-03 09:24, Michael Habibi wrote:
>
>> All,
>>
>> I have interest in creating an image that runs purely in an initramfs
>> space. It seems that overall this shouldn't be too difficult - I can set my
>> IMAGE_FSTYPE to cpio.gz, and as
>> long as it's a reasonable size, it shouldn't be an issue to point to it
>> as an initrd. The only quirk is that kernel modules are split out from the
>> rootfs deployment (two separate
>> archives). Is there a way to combine the modules directly into the rootfs
>> archive?
>>
>
> Just add the kernel-modules package to your install.  This
> will install all of the kernel modules into your rootfs.
>
> --
> 
> Gary Thomas |  Consulting for the
> MLB Associates  |Embedded world
> 
> --
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] do_fetch hanging - any way to see debugging?

[Michael Habibi]

I'm struggling with some network proxy issues and pulling down linux-yocto.
I left my build running overnight and it simply hung at do_fetch. I tried
using -v and -DDD but it doesn't actually show any output from the do_fetch
recipe. I looked at the do_fetch logs and it doesn't have anything useful.
Here are the last lines:

DEBUG: Fetcher failure: Fetch command failed with exit code 8, output:
http://downloads.yoctoproject.org/mirror/sources/git2_git.yoctoproject.org.git.linux-yocto-4.1.git.tar.gz
:
2015-12-02 09:21:15 ERROR 404: Not Found.

DEBUG: Trying Upstream
DEBUG: Fetcher accessed the network with the command git -c
core.fsyncobjectfiles=0 clone --bare --mirror
http://git.yoctoproject.org/git/linux-yocto-4.1.git
/projects/yocto-git/build/downloads/git2/git.yoctoproject.org.git.linux-yocto-4.1.git
DEBUG: Running export
PATH="/projects/yocto-git/scripts:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/x86_64-diags-linux:/projects/yocto-git/build/tmp/sysroots/continental/usr/bin/crossscripts:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/sbin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/sbin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/bin:/projects/yocto-git/scripts:/projects/yocto-git/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/habibim/bin";
export HOME="/home/habibim"; git -c core.fsyncobjectfiles=0 clone --bare
--mirror http://git.yoctoproject.org/git/linux-yocto-4.1.git
/projects/yocto-git/build/downloads/git2/git.yoctoproject.org.git.linux-yocto-4.1.git

I looked at a log where this succeeded, and it didn't have much more
information:

DEBUG: Running export
PATH="/projects/yocto-git/scripts:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/x86_64-poky-linux:/projects/yocto-git/build/tmp/sysroots/genericx86-64/usr/bin/crossscripts:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/sbin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/sbin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/bin:/projects/yocto-git/scripts:/projects/yocto-git/bitbake/bin:/projects/my-distro/scripts:/projects/my-distro/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games";
export HOME="/home/habibim"; git -c core.fsyncobjectfiles=0 branch
--contains aed902160251d69cc28d1e69a4f692e8ea8fa13b --list yocto-4.1 2>
/dev/null | wc -l
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished

Note that I am using a bbappend file for my custom machine:

KBRANCH_continental  = "standard/base"
KMACHINE_continental ?= "common-pc-64"
COMPATIBLE_MACHINE_continental = "continental"

# 4.1.13 rev tag
LINUX_VERSION_continental = "4.1.13"
SRCREV_machine_continental ?= "1f2ce4a2e7aea3a2123b17aff62a80553df31e21"

# Use http protocol
SRC_URI_continental = "git://
git.yoctoproject.org/git/linux-yocto-4.1.git;protocol=http;name=machine;branch=${KBRANCH}
"

If I comment out my overrides for SRCREV and SRC_URI, it simply pulls the
cache (somehow I managed to pull this down before). But if I use my
overrides (to grab a newer kernel, and to use the http protocol), it will
simply hang indefinitely. Any way I can debug what's happening?

Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] do_fetch hanging - any way to see debugging?

[Michael Habibi]

I'll try running it outside the script and see what I can devise - thanks!

On Wed, Dec 2, 2015 at 2:18 PM, Paul Eggleton <paul.eggle...@linux.intel.com
> wrote:

> Hi Michael,
>
> On Wed, 02 Dec 2015 09:41:41 Michael Habibi wrote:
> > I'm struggling with some network proxy issues and pulling down
> linux-yocto.
> > I left my build running overnight and it simply hung at do_fetch. I tried
> > using -v and -DDD but it doesn't actually show any output from the
> do_fetch
> > recipe. I looked at the do_fetch logs and it doesn't have anything
> useful.
> > Here are the last lines:
> >
> > DEBUG: Fetcher failure: Fetch command failed with exit code 8, output:
> >
> http://downloads.yoctoproject.org/mirror/sources/git2_git.yoctoproject.org.g
> > it.linux-yocto-4.1.git.tar.gz
> >
> > 2015-12-02 09:21:15 ERROR 404: Not Found.
> >
> > DEBUG: Trying Upstream
> > DEBUG: Fetcher accessed the network with the command git -c
> > core.fsyncobjectfiles=0 clone --bare --mirror
> > http://git.yoctoproject.org/git/linux-yocto-4.1.git
> >
> /projects/yocto-git/build/downloads/git2/git.yoctoproject.org.git.linux-yoct
> > o-4.1.git DEBUG: Running export
> >
> PATH="/projects/yocto-git/scripts:/projects/yocto-git/build/tmp/sysroots/x86
> >
> _64-linux/usr/bin/x86_64-diags-linux:/projects/yocto-git/build/tmp/sysroots/
> >
> continental/usr/bin/crossscripts:/projects/yocto-git/build/tmp/sysroots/x86_
> >
> 64-linux/usr/sbin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bi
> >
> n:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/sbin:/projects/yocto-g
> >
> it/build/tmp/sysroots/x86_64-linux/bin:/projects/yocto-git/scripts:/projects
> >
> /yocto-git/bitbake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sb
> > in:/bin:/usr/games:/usr/local/games:/home/habibim/bin"; export
> > HOME="/home/habibim"; git -c core.fsyncobjectfiles=0 clone --bare
> --mirror
> > http://git.yoctoproject.org/git/linux-yocto-4.1.git
> >
> /projects/yocto-git/build/downloads/git2/git.yoctoproject.org.git.linux-yoct
> > o-4.1.git
> >
> > I looked at a log where this succeeded, and it didn't have much more
> > information:
> >
> > DEBUG: Running export
> >
> PATH="/projects/yocto-git/scripts:/projects/yocto-git/build/tmp/sysroots/x86
> >
> _64-linux/usr/bin/x86_64-poky-linux:/projects/yocto-git/build/tmp/sysroots/g
> >
> enericx86-64/usr/bin/crossscripts:/projects/yocto-git/build/tmp/sysroots/x86
> >
> _64-linux/usr/sbin:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/b
> >
> in:/projects/yocto-git/build/tmp/sysroots/x86_64-linux/sbin:/projects/yocto-
> >
> git/build/tmp/sysroots/x86_64-linux/bin:/projects/yocto-git/scripts:/project
> >
> s/yocto-git/bitbake/bin:/projects/my-distro/scripts:/projects/my-distro/bitb
> >
> ake/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/ga
> > mes:/usr/local/games"; export HOME="/home/habibim"; git -c
> > core.fsyncobjectfiles=0 branch --contains
> > aed902160251d69cc28d1e69a4f692e8ea8fa13b --list yocto-4.1 2> /dev/null |
> wc
> > -l
> > DEBUG: Python function base_do_fetch finished
> > DEBUG: Python function do_fetch finished
> >
> > Note that I am using a bbappend file for my custom machine:
> >
> > KBRANCH_continental  = "standard/base"
> > KMACHINE_continental ?= "common-pc-64"
> > COMPATIBLE_MACHINE_continental = "continental"
> >
> > # 4.1.13 rev tag
> > LINUX_VERSION_continental = "4.1.13"
> > SRCREV_machine_continental ?= "1f2ce4a2e7aea3a2123b17aff62a80553df31e21"
> >
> > # Use http protocol
> > SRC_URI_continental = "git://
> >
> git.yoctoproject.org/git/linux-yocto-4.1.git;protocol=http;name=machine;bran
> > ch=${KBRANCH} "
> >
> > If I comment out my overrides for SRCREV and SRC_URI, it simply pulls the
> > cache (somehow I managed to pull this down before). But if I use my
> > overrides (to grab a newer kernel, and to use the http protocol), it will
> > simply hang indefinitely. Any way I can debug what's happening?
>
> If it's hung here it may be because the proxy you are behind is trying to
> download and scan the file in its entirety before it allows it to be passed
> through to your download process - we've seen this before. If that's what's
> happening there's not much we can do about that beyond increasing the
> timeout,
> because such proxies usually do not send anything back to the client in the
> way of status. If you need to you c

Re: [yocto] Best practice for files copied directly into sysroot?

[Michael Habibi]

Thanks Bryan!

On Tue, Dec 1, 2015 at 1:50 PM, Bryan Evenson <beven...@melinkcorp.com>
wrote:

> Michael,
>
> > -Original Message-
> > From: yocto-boun...@yoctoproject.org [mailto:yocto-
> > boun...@yoctoproject.org] On Behalf Of Michael Habibi
> > Sent: Tuesday, December 01, 2015 10:26 AM
> > To: yocto@yoctoproject.org
> > Subject: [yocto] Best practice for files copied directly into sysroot?
> >
> > I am working on potentially migrating our distribution to Yocto (will be
> a long,
> > long process). We have an area of our filesystem where any file placed
> will go
> > directly into the rootfs of the target. This works well with our
> product, as we
> > have certain binaries and scripts that need to be copied directly, and
> there's
> > no other real step involved (no compilation, etc). Is there a best
> practice or
> > existing class that handles this? I saw mention of a binary/ directory
> but I
> > wasn't sure if that was for this or something else.
> >
> > Thanks,
> > Michael
>
> I'm glad I decided to answer your question because I think I found an
> easier way to do it than I have been doing this in the past.  If you want
> to be able to update the files on your system at a later date, you will
> want to make your own recipe which installs the binaries and scripts onto
> the rootfs.  In general you create a recipe which does not have
> do_configure or do_compile tasks.  There is a class that does this already
> called "bin_package" that does this for you.  See the "Packaging Externally
> Produced Binaries" section of the manual here:
> http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html#new-recipe-testing-examples.
>  So if you include the line "inherit bin_package" and follow the
> instructions in that section of the manual, it should install all the files
> to the rootfs that are included in the recipe.
>
> Regards,
> Bryan
>
>
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

[yocto] Best practice for files copied directly into sysroot?

[Michael Habibi]

I am working on potentially migrating our distribution to Yocto (will be a
long, long process). We have an area of our filesystem where any file
placed will go directly into the rootfs of the target. This works well with
our product, as we have certain binaries and scripts that need to be copied
directly, and there's no other real step involved (no compilation, etc). Is
there a best practice or existing class that handles this? I saw mention of
a binary/ directory but I wasn't sure if that was for this or something
else.

Thanks,
Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Mark, I am continuing to look at this. I hope you don't mind if I keep
updating this thread with my investigation.

First, I figured out my confusion. I was associating the wrong Makefile
command with the illegal instruction. It was actually the command under the
recipe for 'sharedmods', but it was silenced in the Makefile so I assumed
it was another line. You're right that the path with the *.so's is the
offending path. I've scoured online patches and can't understand how this
mechanism works outside of Yocto. I figure people trying to compile for ARM
would have similar issues. There seems to be a fundamental problem with how
Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
However, other people seem to have successfully built python for other
architectures, so I'm still confused.

I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory, and
the python recipe successfully passed. Can you elaborate on the issues you
were having with your patch? The only difference is that I maintained the
other two directories in $PYTHONPATH,
e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
offending one.

On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <mikehab...@gmail.com>
wrote:

> Mark,
>
> I ran the same command, including the offending PYTHONPATH, from the shell
> without error. Any reason why that would work, despite it not working from
> within the Makefile? I was thinking it was another environment variable
> that was causing the issue (one that's not set in my bash environment by
> default). My $PATH didn't have python-native so I called it using an
> absolute path, but otherwise it's the same command run from the Makefile
> (parsed, of course):
>
> bash$
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> _PYTHON_HOST_PLATFORM=linux2-x86_64
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
> -S -m sysconfig --generate-posix-vars
> bash$ ls pybuilddir.txt
> pybuilddir.txt
>
> Maybe I'm missing something.
>
>
> On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.ha...@windriver.com>
> wrote:
>
>> On 11/25/15 10:11 AM, Michael Habibi wrote:
>> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing
>> to a
>> > native version of Python:
>> >
>> > *| which python2.7*
>> > *|
>> >
>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
>> > |
>> >
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> > _PYTHON_HOST_PLATFORM=linux2-x86_64
>> >
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> > python2.7 -S -m sysconfig --generate-posix-vars ;\
>> > |   if test $? -ne 0 ; then \
>> > |   echo "generate-posix-vars failed" ; \
>> > |   rm -f ./pybuilddir.txt ; \
>> > |   exit 1 ; \
>> > |   fi
>> > | Illegal instruction (core dumped)
>> > | make: *** [sharedmods] Error 132
>> >
>> > I guess one of the environmental variables being passed in is screwing
>> up what
>> > it's doing. Unfortunately I don't know enough about the inner workings
>> of Python
>> > to be of much help moving forward. I gave it my best shot!
>>
>> The problem I tracked down was:
>>
>>
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>>
>> This is full of .so objects when loaded cause the illegal instruction.
>>
>> I came up with a patch that I thought was going to fix it, but it's
>> triggered
>> other failures.
>>
>>
>> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>>
>> Just to be clear.. this does NOT work properly in many cases.  But might
>> give
>> you or someone else a clue as to how to possibly fix it.
>>
>> --Mark
>>
>> >
>> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehab...

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

OK I've dug into this further today, and here are some findings:

With my patch (which is similar to yours), it fails do_compile/package_qa
because of poisoned system directories during the compilation steps. This
approach seems to be a bust. Originally when I tested it, I hacked up the
Makefile and only re-ran do_install onward, and didn't re-run do_compile
(because I had to create the patch). Once I created the patch and re-ran
from clean, I got some errors.

It appears the only reason it is using that directory in PYTHONPATH at all
(the one generated from the shell command and pybuilddir.txt) is for a
potential sysconfig file in there. I tracked it down to a comment in a bug
on Python's bug tracker:

http://bugs.python.org/issue15484#msg180577

If we change that directory, it will pull in the sysconfig settings from
the native Python configure (probably defaults to native sysconfig settings
because the PATH doesn't include the overrides), and then do some really
bad things. The question then became: in the unmodified build, why does
do_install fail but do_compile doesn't? They both seem to be using
sharedmods recipe which uses $PYTHONPATH. The answer is that when we run
do_compile the first time around, the offending *.so's are not generated
yet, and thus are not included by python-native. The second time around (in
do_install), the *.so's are in the path and so python-native will use them
and override its own.

My latest experiment is to remove that location altogether from PYTHONPATH
and see if it works (neither the shell-generated directory in python-cross,
nor the lib-dynload directory from python-native). The compile works the
first time through despite there being no sysconfig data, so I'm wondering
if the install step will also work without finding any data. python-native
should continue to work with its internal settings to find modules and
libraries.

To be continued tomorrow, I'm sure...

On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi <mikehab...@gmail.com>
wrote:

> Mark, I am continuing to look at this. I hope you don't mind if I keep
> updating this thread with my investigation.
>
> First, I figured out my confusion. I was associating the wrong Makefile
> command with the illegal instruction. It was actually the command under the
> recipe for 'sharedmods', but it was silenced in the Makefile so I assumed
> it was another line. You're right that the path with the *.so's is the
> offending path. I've scoured online patches and can't understand how this
> mechanism works outside of Yocto. I figure people trying to compile for ARM
> would have similar issues. There seems to be a fundamental problem with how
> Python sets up PYTHONPATH using the pybuilddir file and $abs_builddir.
> However, other people seem to have successfully built python for other
> architectures, so I'm still confused.
>
> I hardcoded a fix for PYTHONPATH to use the host lib-dynload directory,
> and the python recipe successfully passed. Can you elaborate on the issues
> you were having with your patch? The only difference is that I maintained
> the other two directories in $PYTHONPATH,
> e.g. $(srcdir)/Lib:$(srcdir)/Lib/plat-$(MACHDEP), and only replaced the
> offending one.
>
> On Wed, Nov 25, 2015 at 4:58 PM, Michael Habibi <mikehab...@gmail.com>
> wrote:
>
>> Mark,
>>
>> I ran the same command, including the offending PYTHONPATH, from the
>> shell without error. Any reason why that would work, despite it not working
>> from within the Makefile? I was thinking it was another environment
>> variable that was causing the issue (one that's not set in my bash
>> environment by default). My $PATH didn't have python-native so I called it
>> using an absolute path, but otherwise it's the same command run from the
>> Makefile (parsed, of course):
>>
>> bash$
>> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
>> _PYTHON_HOST_PLATFORM=linux2-x86_64
>> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
>> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
>> -S -m sysconfig --generate-posix-vars
>> bash$ ls pybuilddir.txt
>> pybuilddir.txt
>>
>> Maybe I'm missing something.
>>
>>
>> On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.ha...@windriver.com>
>> wrote:
>>
>>> On 11/25/15 10:11 AM, Michael Habibi wrote:
>>> > Well, I'm wrong yet again. A 'which python2.7' shows that it's
>>> pointing to a
>>> 

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Mark, great! I was literally about to try double-stacking the PATH next
(since removing both didn't work for reasons that became obvious after
running the experiment), so that it would grab the *.so's from
python-native and then the sysconfig from the python-cross. Looks like I
may need to subscribe to that list so I'm aware of these patches!

I'm not sure what the streams look like (I can't visualize it yet), but is
this change in any Yocto branch yet, or does it need to get pulled by
someone? I can merge it locally, but just curious how the OE-Core and Yocto
repos are related.

On Mon, Nov 30, 2015 at 6:07 PM, Mark Hatle <mark.ha...@windriver.com>
wrote:

> This is the patch that we ended up using:
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113457.html
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113456.html
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113458.html
>
> On 11/30/15 5:12 PM, Michael Habibi wrote:
> > OK I've dug into this further today, and here are some findings:
> >
> > With my patch (which is similar to yours), it fails do_compile/package_qa
> > because of poisoned system directories during the compilation steps. This
> > approach seems to be a bust. Originally when I tested it, I hacked up the
> > Makefile and only re-ran do_install onward, and didn't re-run do_compile
> > (because I had to create the patch). Once I created the patch and re-ran
> from
> > clean, I got some errors.
> >
> > It appears the only reason it is using that directory in PYTHONPATH at
> all (the
> > one generated from the shell command and pybuilddir.txt) is for a
> potential
> > sysconfig file in there. I tracked it down to a comment in a bug on
> Python's bug
> > tracker:
> >
> > http://bugs.python.org/issue15484#msg180577
> >
> > If we change that directory, it will pull in the sysconfig settings from
> the
> > native Python configure (probably defaults to native sysconfig settings
> because
> > the PATH doesn't include the overrides), and then do some really bad
> things. The
> > question then became: in the unmodified build, why does do_install fail
> but
> > do_compile doesn't? They both seem to be using sharedmods recipe which
> uses
> > $PYTHONPATH. The answer is that when we run do_compile the first time
> around,
> > the offending *.so's are not generated yet, and thus are not included by
> > python-native. The second time around (in do_install), the *.so's are in
> the
> > path and so python-native will use them and override its own.
> >
> > My latest experiment is to remove that location altogether from
> PYTHONPATH and
> > see if it works (neither the shell-generated directory in python-cross,
> nor the
> > lib-dynload directory from python-native). The compile works the first
> time
> > through despite there being no sysconfig data, so I'm wondering if the
> install
> > step will also work without finding any data. python-native should
> continue to
> > work with its internal settings to find modules and libraries.
> >
> > To be continued tomorrow, I'm sure...
> >
> > On Mon, Nov 30, 2015 at 11:37 AM, Michael Habibi <mikehab...@gmail.com
> > <mailto:mikehab...@gmail.com>> wrote:
> >
> > Mark, I am continuing to look at this. I hope you don't mind if I
> keep
> > updating this thread with my investigation.
> >
> > First, I figured out my confusion. I was associating the wrong
> Makefile
> > command with the illegal instruction. It was actually the command
> under the
> > recipe for 'sharedmods', but it was silenced in the Makefile so I
> assumed it
> > was another line. You're right that the path with the *.so's is the
> > offending path. I've scoured online patches and can't understand how
> this
> > mechanism works outside of Yocto. I figure people trying to compile
> for ARM
> > would have similar issues. There seems to be a fundamental problem
> with how
> > Python sets up PYTHONPATH using the pybuilddir file and
> $abs_builddir.
> > However, other people seem to have successfully built python for
> other
> > architectures, so I'm still confused.
> >
> > I hardcoded a fix for PYTHONPATH to use the host lib-dynload
> directory, and
> > the python recipe successfully passed. Can you elaborate on the
> issues you
> > were having with your patch? The only difference is that I
> maintained the
> > other two directories in $PYTHONPATH,
> > e.g. $(srcdir)

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

For what it's worth, this is the offending portion of Makefile.pre:452:

# Create build directory and generate the sysconfig build-time data there.
# pybuilddir.txt contains the name of the build dir and is used for
# sys.path fixup -- see Modules/getpath.c.
# Since this step runs before shared modules are built, try to avoid
bootstrap
# problems by creating a dummy pybuilddir.txt just to allow interpreter
# initialization to succeed.  It will be overwritten by generate-posix-vars
# or removed in case of failure.
pybuilddir.txt: $(BUILDPYTHON)
→ @echo "none" > ./pybuilddir.txt
→ $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars ;\
→ if test $$? -ne 0 ; then \
→ → echo "generate-posix-vars failed" ; \
→ → rm -f ./pybuilddir.txt ; \
→ → exit 1 ; \
→ fi

I don't know enough about the Python build to understand what it's trying
to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
acceptable?

I'm surprised this seems to work for other people, since this should be
failing for anyone using Python on a target platform different from their
host.

On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <mark.ha...@windriver.com>
wrote:

> On 11/24/15 3:23 PM, Mark Hatle wrote:
> > My guess is that there is a bug in the python integration where it's not
> > realizing the host and target are different systems, so it's trying to
> run a
> > target program on your host.  Your host isn't haswell, so... Illegal
> Instruction
> > -- and the system stops.
>
> Just an FYI, I hit the same problem today.  I suspect it is the host
> trying to
> run target software and I'm looking into it.
>
> --Mark
>
> > The alternative would be something is running QEMU to execute a target
> binary
> > and QEMU doesn't have instruction support -- but that doesn't look like
> the case
> > here.
> >
> > --Mark
> >
> > On 11/24/15 3:06 PM, Michael Habibi wrote:
> >> All,
> >>
> >> I added a new machine definition with tuning parameters for haswell
> >> microarchitectures (basically just duplicated corei7 but tuned it for
> haswell).
> >> This seems to work correctly, but failed when running do_install for
> python
> >> toward the end of the build process. I am running with the Yocto 2.0
> framework,
> >> with very minimal changes to create a new distribution and machine for
> our
> >> custom embedded device. Note I had this distro configuration working
> before, and
> >> the only difference is I added a new machine with this tuning.
> >>
> >> I believe the issue is because, as part of the do_install step for
> Python, it
> >> attempts to run python on the local host, despite being cross-compiled.
> However,
> >> it is tuned for a processor that my host machine doesn't run, so I get
> an
> >> instruction exception.
> >>
> >> Did I do something weird? I figure python would be configured for
> >> cross-compiling and therefore wouldn't try to run the target version on
> the
> >> host, even for sanity tests. Here is the output of the failure:
> >>
> >> $ tail -20
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/temp/log.do_install.17258
> >>
> >> x86_64-diags-linux-ar rc libpython2.7.a Modules/threadmodule.o
> >>  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o
> >>  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o
> >>  Modules/_weakref.o  Modules/zipimport.o  Modules/symtablemodule.o
> >>  Modules/md5module.o Modules/md5.o  Modules/xxsubtype.o
> >> x86_64-diags-linux-ranlib libpython2.7.a
> >> Modules/posixmodule.o: In function `posix_tmpnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7575:
> >> warning: the use of `tmpnam_r' is dangerous, better use `mkstemp'
> >> Modules/posixmodule.o: In function `posix_tempnam':
> >>
> /projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Modules/posixmodule.c:7522:
> >> warning: the use of `tempnam' is dangerous, better use `mkstemp'
> >> x86_64-diags-linux-gcc  -m64 -march=haswell -mtune=haswell -mfpmath=sse
> -mavx2
> >> --sysroot=/projects/yocto-git/build/tmp/sysroots/continental -Wl,-O1
> >> -Wl,--hash-style=gnu -Wl,--as-needed -Xlinker -export-dynamic -o python
> \
> >> Modules/python.o \
> >> -L. -lpython2.7 -lpthread -ldl  -lpthread
> -lutil   -lm
> >>
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing to
a native version of Python:

*| which python2.7*
*|
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
|
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
python2.7 -S -m sysconfig --generate-posix-vars ;\
|   if test $? -ne 0 ; then \
|   echo "generate-posix-vars failed" ; \
|   rm -f ./pybuilddir.txt ; \
|   exit 1 ; \
|   fi
| Illegal instruction (core dumped)
| make: *** [sharedmods] Error 132

I guess one of the environmental variables being passed in is screwing up
what it's doing. Unfortunately I don't know enough about the inner workings
of Python to be of much help moving forward. I gave it my best shot!


On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehab...@gmail.com>
wrote:

> Ah sorry, I misspoke. I walked through the Makefile and configure scripts
> a bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> Python. It seems it's not being configured correctly. Either the path is
> setup wrong and python2.7 is not pointing to the right version, or
> python2.7 needs to be pointing to the absolute path instead of relying on
> $PATH.
>
>
>
>
> On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <mikehab...@gmail.com>
> wrote:
>
>> For what it's worth, this is the offending portion of Makefile.pre:452:
>>
>> # Create build directory and generate the sysconfig build-time data there.
>> # pybuilddir.txt contains the name of the build dir and is used for
>> # sys.path fixup -- see Modules/getpath.c.
>> # Since this step runs before shared modules are built, try to avoid
>> bootstrap
>> # problems by creating a dummy pybuilddir.txt just to allow interpreter
>> # initialization to succeed.  It will be overwritten by
>> generate-posix-vars
>> # or removed in case of failure.
>> pybuilddir.txt: $(BUILDPYTHON)
>> → @echo "none" > ./pybuilddir.txt
>> → $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars
>> ;\
>> → if test $$? -ne 0 ; then \
>> → → echo "generate-posix-vars failed" ; \
>> → → rm -f ./pybuilddir.txt ; \
>> → → exit 1 ; \
>> → fi
>>
>> I don't know enough about the Python build to understand what it's trying
>> to do, but perhaps replacing PYTHON_FOR_BUILD with HOSTPYTHON may be
>> acceptable?
>>
>> I'm surprised this seems to work for other people, since this should be
>> failing for anyone using Python on a target platform different from their
>> host.
>>
>> On Wed, Nov 25, 2015 at 9:02 AM, Mark Hatle <mark.ha...@windriver.com>
>> wrote:
>>
>>> On 11/24/15 3:23 PM, Mark Hatle wrote:
>>> > My guess is that there is a bug in the python integration where it's
>>> not
>>> > realizing the host and target are different systems, so it's trying to
>>> run a
>>> > target program on your host.  Your host isn't haswell, so... Illegal
>>> Instruction
>>> > -- and the system stops.
>>>
>>> Just an FYI, I hit the same problem today.  I suspect it is the host
>>> trying to
>>> run target software and I'm looking into it.
>>>
>>> --Mark
>>>
>>> > The alternative would be something is running QEMU to execute a target
>>> binary
>>> > and QEMU doesn't have instruction support -- but that doesn't look
>>> like the case
>>> > here.
>>> >
>>> > --Mark
>>> >
>>> > On 11/24/15 3:06 PM, Michael Habibi wrote:
>>> >> All,
>>> >>
>>> >> I added a new machine definition with tuning parameters for haswell
>>> >> microarchitectures (basically just duplicated corei7 but tuned it for
>>> haswell).
>>> >> This seems to work correctly, but failed when running do_install for
>>> python
>>> >> toward the end of the build process. I am running with the Yocto 2.0
>>> framework,
>>> >> with very minimal changes to create a new distribution and machine
>>> for our
>>> >> custom embedded device. Note I had this distro configuration working
>>> before, and
>>> >>

Re: [yocto] Cross-compiling python fails with new custom tuning

[Michael Habibi]

Mark,

I ran the same command, including the offending PYTHONPATH, from the shell
without error. Any reason why that would work, despite it not working from
within the Makefile? I was thinking it was another environment variable
that was causing the issue (one that's not set in my bash environment by
default). My $PATH didn't have python-native so I called it using an
absolute path, but otherwise it's the same command run from the Makefile
(parsed, of course):

bash$
_PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
_PYTHON_HOST_PLATFORM=linux2-x86_64
PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
/projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7
-S -m sysconfig --generate-posix-vars
bash$ ls pybuilddir.txt
pybuilddir.txt

Maybe I'm missing something.


On Wed, Nov 25, 2015 at 4:38 PM, Mark Hatle <mark.ha...@windriver.com>
wrote:

> On 11/25/15 10:11 AM, Michael Habibi wrote:
> > Well, I'm wrong yet again. A 'which python2.7' shows that it's pointing
> to a
> > native version of Python:
> >
> > *| which python2.7*
> > *|
> >
> /projects/yocto-git/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7*
> > |
> >
> _PYTHON_PROJECT_BASE=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build
> > _PYTHON_HOST_PLATFORM=linux2-x86_64
> >
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib:/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/Python-2.7.9/Lib/plat-linux2
> > python2.7 -S -m sysconfig --generate-posix-vars ;\
> > |   if test $? -ne 0 ; then \
> > |   echo "generate-posix-vars failed" ; \
> > |   rm -f ./pybuilddir.txt ; \
> > |   exit 1 ; \
> > |   fi
> > | Illegal instruction (core dumped)
> > | make: *** [sharedmods] Error 132
> >
> > I guess one of the environmental variables being passed in is screwing
> up what
> > it's doing. Unfortunately I don't know enough about the inner workings
> of Python
> > to be of much help moving forward. I gave it my best shot!
>
> The problem I tracked down was:
>
>
> PYTHONPATH=/projects/yocto-git/build/tmp/work/haswell-diags-linux/python/2.7.9-r1/build/build/lib.linux2-x86_64-2.7
>
> This is full of .so objects when loaded cause the illegal instruction.
>
> I came up with a patch that I thought was going to fix it, but it's
> triggered
> other failures.
>
>
> http://lists.openembedded.org/pipermail/openembedded-core/2015-November/113341.html
>
> Just to be clear.. this does NOT work properly in many cases.  But might
> give
> you or someone else a clue as to how to possibly fix it.
>
> --Mark
>
> >
> > On Wed, Nov 25, 2015 at 10:01 AM, Michael Habibi <mikehab...@gmail.com
> > <mailto:mikehab...@gmail.com>> wrote:
> >
> > Ah sorry, I misspoke. I walked through the Makefile and configure
> scripts a
> > bit more, and $(PYTHON_FOR_BUILD) should in fact be a host version of
> > Python. It seems it's not being configured correctly. Either the
> path is
> > setup wrong and python2.7 is not pointing to the right version, or
> python2.7
> > needs to be pointing to the absolute path instead of relying on
> $PATH.
> >
> >
> >
> >
> > On Wed, Nov 25, 2015 at 9:45 AM, Michael Habibi <
> mikehab...@gmail.com
> > <mailto:mikehab...@gmail.com>> wrote:
> >
> > For what it's worth, this is the offending portion of
> Makefile.pre:452:
> >
> > # Create build directory and generate the sysconfig build-time
> data there.
> > # pybuilddir.txt contains the name of the build dir and is used
> for
> > # sys.path fixup -- see Modules/getpath.c.
> > # Since this step runs before shared modules are built, try to
> avoid
> > bootstrap
> > # problems by creating a dummy pybuilddir.txt just to allow
> interpreter
> > # initialization to succeed.  It will be overwritten by
> generate-posix-vars
> > # or removed in case of failure.
> > pybuilddir.txt: $(BUILDPYTHON)
> > → @echo "none" > ./pybuilddir.txt
> > → $(RUNSHARE

Re: [yocto] Transitioning to Yocto, some basic questions about workflow

[Michael Habibi]

Randy, I've had some time to think about your post and I had a follow up
question. Let's say we had internal applications, you suggest that the
fetcher can be used to pull down the git/svn repos that contains the
internal application code. In this environment, where is the Yocto
framework hosted? Is it also on a repo? Would then we have a Yocto repo for
image building, and an application repo for application development? Is
there a clean way of combining both Yocto and internal applications into a
single repo, so our engineers simply have to do a 'git clone' and they'll
have what they need to get started?

Thanks, still trying to wrap my head around this a bit.

On Tue, Nov 10, 2015 at 6:06 PM, Randy Witt <randy.e.w...@linux.intel.com>
wrote:

> Hi Michael,
>
> See my replies below.
>
> On 11/10/2015 02:27 PM, Michael Habibi wrote:
>
>
>>
>> This is fundamentally why I have been looking into Yocto/BB/OE as a
>> potential replacement distribution. However, I have a couple questions
>> stemming from my research. We can leverage the existing Yocto build for
>> various open source utilities where it pulls from the web, patches,
>> builds,
>> and installs into the deployable image. We would probably want to colocate
>> the tarballs locally, because we would like to prevent people from having
>> to fetch from the web during build cycles.
>>
>>
> This is pretty common. You can set up SOURCE_MIRROR_URL, to specify local
> mirrors to try before using the upstream url in the recipe. So each person
> building would at least have to fetch from the local mirror for a build,
> but subsequent builds wouldn't require fetching again as long as the DL_DIR
> was preserved.
>
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SOURCE_MIRROR_URL
>
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DL_DIR
>
>
>> I see that Yocto thinks of the deployable image and applications as
>> separate entities: first you'd build the distribution, then you would use
>> ADT/cross-toolchain/etc to build the applications and install them
>> separately. However, in our environment, we would need all of our custom
>> IP
>> applications to be built as part of the deployable image, and not as a
>> separate procdure. For example, if I were to call "bitbake our-image", I
>> would like the deployable image to contain 1) the kernel, 2) various core
>> utilities and libraries for booting, and 3) our custom applications for
>> our
>> device.
>>
>>
>> This means that somewhere in the yocto framework, we'd ideally have some
>> source code somewhere that would also be compiled via recipes/classes that
>> we'd have to custom write. Is there a best practice for this kind of
>> workflow? I don't mind not having source checked in to our VCS for things
>> like the kernel, OpenSSL, etc (those can be tarballs obtained from a local
>> server), but we likely wouldn't want to host tarballs for the applications
>> we are writing and modifying day-to-day by dozens of engineers. Is there a
>> place where this source would best fit? Would it be under build/tmp/etc,
>> or
>> perhaps we can locate the source under a layer directory, like
>> meta/source/our-ip-applications?
>>
>>
> For all the internal applications at my previous employer we had a layer
> for all internal items, it's quite common.
>
> And if you're not aware, source code is not restricted to tarballs. You
> can use git repos, svn and any mechanism the fetcher supports.(even local
> directories) The following URL lists the types the fetcher supports.
>
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SRC_URI
>
> The workflow now encouraged, is to use recipetool to assist in creating
> recipes. And then once the recipes are created, you can use devtool to
> actually do the iterative development on the application represented by the
> recipe.
>
> https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow
>
>
>> Or perhaps this workflow is just not recommended or supported by the Yocto
>> Project? If that's the case, does Yocto only recommend building the
>> distribution then building the applications as completely separate
>> workflows? Or is there another workflow that I haven't stumbled across
>> yet?
>>
>>
> There is nothing "wrong" with building the image each time as a developer.
> However, it is time consuming to construct the full image and deploy it.
>
> A much more appealing mechanism would be to use devtool to build the
> recipe you are working on, and then use "devtool deploy

[yocto] Transitioning to Yocto, some basic questions about workflow

[Michael Habibi]

All,



First I appreciate taking the time to read through this email. I have spent
the last couple weeks reading through Yocto/bitbake documentation, and have
run a couple of the reference builds, e.g. core-image-minimal. I have a
better-than-vague sense now of how it’s parsing confs/recipes/classes and
putting that together.


My understanding is that Yocto is nothing more than a reference build that
combines OE and BitBake to give us a good starting point for a custom
distribution. Hopefully that assumption is correct.


Our current build distribution at work is a monstrosity. We use a
proprietary distribution that combines their utilities that they provide,
along with various other open source utilities and the linux kernel.
Overall our source tree is on the order of several gig in size, with a
couple hundred build components. The build framework is close to buildroot
in that we just iterate through directories, build the applications (in
some cases it's source, in other cases it's a patched tarball), and build
up a root filesystem image that we install on the embedded target. We have
many challenges with this, namely with dependencies and build efficiencies
- we can't run parallel builds because the way it iterates through the
makefiles is not dependency-based, but directory-based.


This is fundamentally why I have been looking into Yocto/BB/OE as a
potential replacement distribution. However, I have a couple questions
stemming from my research. We can leverage the existing Yocto build for
various open source utilities where it pulls from the web, patches, builds,
and installs into the deployable image. We would probably want to colocate
the tarballs locally, because we would like to prevent people from having
to fetch from the web during build cycles.


I see that Yocto thinks of the deployable image and applications as
separate entities: first you'd build the distribution, then you would use
ADT/cross-toolchain/etc to build the applications and install them
separately. However, in our environment, we would need all of our custom IP
applications to be built as part of the deployable image, and not as a
separate procdure. For example, if I were to call "bitbake our-image", I
would like the deployable image to contain 1) the kernel, 2) various core
utilities and libraries for booting, and 3) our custom applications for our
device.


This means that somewhere in the yocto framework, we'd ideally have some
source code somewhere that would also be compiled via recipes/classes that
we'd have to custom write. Is there a best practice for this kind of
workflow? I don't mind not having source checked in to our VCS for things
like the kernel, OpenSSL, etc (those can be tarballs obtained from a local
server), but we likely wouldn't want to host tarballs for the applications
we are writing and modifying day-to-day by dozens of engineers. Is there a
place where this source would best fit? Would it be under build/tmp/etc, or
perhaps we can locate the source under a layer directory, like
meta/source/our-ip-applications?


Or perhaps this workflow is just not recommended or supported by the Yocto
Project? If that's the case, does Yocto only recommend building the
distribution then building the applications as completely separate
workflows? Or is there another workflow that I haven't stumbled across yet?


Thanks again for all your help, and let me know if I can help clarify
anything,


Michael
-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto

Re: [yocto] Transitioning to Yocto, some basic questions about workflow

[Michael Habibi]

Thanks for the great info Randy.

On Tue, Nov 10, 2015 at 6:06 PM, Randy Witt <randy.e.w...@linux.intel.com>
wrote:

> Hi Michael,
> See my replies below.
> On 11/10/2015 02:27 PM, Michael Habibi wrote:
>>
>>
>> This is fundamentally why I have been looking into Yocto/BB/OE as a
>> potential replacement distribution. However, I have a couple questions
>> stemming from my research. We can leverage the existing Yocto build for
>> various open source utilities where it pulls from the web, patches, builds,
>> and installs into the deployable image. We would probably want to colocate
>> the tarballs locally, because we would like to prevent people from having
>> to fetch from the web during build cycles.
>>
> This is pretty common. You can set up SOURCE_MIRROR_URL, to specify local 
> mirrors to try before using the upstream url in the recipe. So each person 
> building would at least have to fetch from the local mirror for a build, but 
> subsequent builds wouldn't require fetching again as long as the DL_DIR was 
> preserved.
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SOURCE_MIRROR_URL
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-DL_DIR
>>
>> I see that Yocto thinks of the deployable image and applications as
>> separate entities: first you'd build the distribution, then you would use
>> ADT/cross-toolchain/etc to build the applications and install them
>> separately. However, in our environment, we would need all of our custom IP
>> applications to be built as part of the deployable image, and not as a
>> separate procdure. For example, if I were to call "bitbake our-image", I
>> would like the deployable image to contain 1) the kernel, 2) various core
>> utilities and libraries for booting, and 3) our custom applications for our
>> device.
>>
>>
>> This means that somewhere in the yocto framework, we'd ideally have some
>> source code somewhere that would also be compiled via recipes/classes that
>> we'd have to custom write. Is there a best practice for this kind of
>> workflow? I don't mind not having source checked in to our VCS for things
>> like the kernel, OpenSSL, etc (those can be tarballs obtained from a local
>> server), but we likely wouldn't want to host tarballs for the applications
>> we are writing and modifying day-to-day by dozens of engineers. Is there a
>> place where this source would best fit? Would it be under build/tmp/etc, or
>> perhaps we can locate the source under a layer directory, like
>> meta/source/our-ip-applications?
>>
> For all the internal applications at my previous employer we had a layer for 
> all 
> internal items, it's quite common.
> And if you're not aware, source code is not restricted to tarballs. You can 
> use 
> git repos, svn and any mechanism the fetcher supports.(even local 
> directories) 
> The following URL lists the types the fetcher supports.
> https://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#var-SRC_URI
> The workflow now encouraged, is to use recipetool to assist in creating 
> recipes. 
> And then once the recipes are created, you can use devtool to actually do the 
> iterative development on the application represented by the recipe.
> https://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#using-devtool-in-your-workflow
>>
>> Or perhaps this workflow is just not recommended or supported by the Yocto
>> Project? If that's the case, does Yocto only recommend building the
>> distribution then building the applications as completely separate
>> workflows? Or is there another workflow that I haven't stumbled across yet?
>>
> There is nothing "wrong" with building the image each time as a developer. 
> However, it is time consuming to construct the full image and deploy it.
> A much more appealing mechanism would be to use devtool to build the recipe 
> you 
> are working on, and then use "devtool deploy-target" to then deploy the 
> output 
> to a live machine. This would require a writeable filesystem on the target 
> device, but if that is available, the workflow and turnaround time is much 
> faster.
>>
>> Thanks again for all your help, and let me know if I can help clarify
>> anything,
>>
>>
>> Michael
>>
>>
>>-- 
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto