Re: [yocto] tar ball vs. git development questions
On Jan 23, 2012, at 9:01 AM, Bruce Ashfield wrote:
On 12-01-23 08:10 AM, Gary Thomas wrote:
On 2012-01-23 05:51, jfabernathy wrote:
On 01/22/2012 08:12 PM, Gary Thomas wrote:
On 2012-01-22 13:19, James Abernathy wrote:
I have used both git and the tarball methods of bitbaking projects,
all of them derivatives of the examples in the Yocto documentation.
I was having issues using the local clone of
the Yocto kernel git repository this weekend. I had successfully
done that before, but I was rebuilding the PC workstation, and
getting everything setup and tested some of the
meta-intel BSPs to make sure I had everything right. Cloning the
linux-yocto-3.0 repository was successful, but the bakes against it
failed. I made sure I had poky-extras setup
right, but I still had problems. To isolate the problem, I changed
to building with the tarballs and everything worked fine.
So that got me thinking what are the differences between the 2 methods:
* I assume that if I use the tarball method, bitbake, using the
recipes, pulls down files from the online repositories and puts
those files into the centralized local download
directory ($DL_DIR), allowing reuse instead of re-downloading each
time. The content downloaded for linux-yocto-3.0 is exactly what
would be pulled from the local repository if
I used a local clone of the git repository for linux-yocto-3.0.
* If my assumption above is correct, if I'm not modifying the source
code of the kernel (only changing config parameters), then once
you've run at least one build with the
tarball method, the $DL_DIR directory contains all the files you'll
need to build any image with linux-yocto-3.0. So there is no need to
have a local clone of the kernel
repository for speeding up development. Am I right?
* If I have a successful creation of a bare clone of
linux-yocto-3.0.git, how could builds of Edison packages be failing?
That makes me concerned about using git and successfully
repeating builds of stable branches like Edison.
If you set BB_GENERATE_MIRROR_TARBALLS = 1 (e.g. in local.conf)
then you'll get tarballs which hold the git repositories after
download. You can then reuse these (by sharing the DL_DIR or
using a local mirror). Does that help with the issue you're seeing?
I'm not sure it does. I don't want poky to do more work. I have my
download directory, $DL_DIR, outside my build directory so I can keep
it run to run, as mentioned in the comments
in local.conf. I was trying to understand 2 basic questions:
1. what could be causing build failures using a freshly created bare
clone yesterday vs. using the poky-edison-6.0 tarball. It would be
scary if you could clone the linux-yocto-3.0
successfully one day and have it be used in a build successfully, but
clone it another day and it not work. I figured that bitbake/poky
pulled the same information into DL_DIR
regardless of whether you pulled from the bare clone locally or
straight from the on-line repository.
What kind of errors? Some details might help understand what the problem
is.
2. I was trying to look at items to speed up build runs. I thought
that if the downloads in DL_DIR were reused if they existed, it would
speed up a run. It seems to. So the
question is, after the first build, the files I need for
linux-yocto-3.0 are in DL_DIR regardless of whether they came from the
online repository or the local bare clone. right?
I think so, but I'm not 100% sure.
From my experience, this is true. Regardless of the SRCI_URI, the
downloads directory contains everything that poky/bitbake need to
do builds after the initial fetch has been performed. And that's
where subsequent updates are pullled.
Cheers,
Bruce
I'm going to retest all this and try some of these tricks when I have some
time. If I get any more errors with the local repository, I'll post them.
Thanks, Jim A
The way I have my system set up, I never download anything more
than once :-) I have a shared source repository which I point to
using MIRRORS and then all my builds are relative to that. If there
is new code, it gets downloaded and saved (as a tarball) in my sources
repository to be used by subsequent builds. To do this, I just have
these lines in distro.conf
# Provide pre-staged sources
SOURCE_MIRROR_URL ?= file://${COREBASE}/sources/
INHERIT += own-mirrors
BB_GENERATE_MIRROR_TARBALLS = 1
I don't define DL_DIR in local.conf so files only come from my mirror.
This process is so successful that I almost always run with
BB_NO_NETWORK=1
which causes the fetcher to die if it can't find the file locally. If I
find
that I do need to download something new, I disable this and let the
fetcher
download it. I'll then have a saved tarball (either downloaded directly or
synthesized) in build_dir/downloads that I can push to my mirror.
___
yocto mailing list
yocto@yoctoproject.org
Re: [yocto] tar ball vs. git development questions
On 01/22/2012 08:12 PM, Gary Thomas wrote: On 2012-01-22 13:19, James Abernathy wrote: I have used both git and the tarball methods of bitbaking projects, all of them derivatives of the examples in the Yocto documentation. I was having issues using the local clone of the Yocto kernel git repository this weekend. I had successfully done that before, but I was rebuilding the PC workstation, and getting everything setup and tested some of the meta-intel BSPs to make sure I had everything right. Cloning the linux-yocto-3.0 repository was successful, but the bakes against it failed. I made sure I had poky-extras setup right, but I still had problems. To isolate the problem, I changed to building with the tarballs and everything worked fine. So that got me thinking what are the differences between the 2 methods: * I assume that if I use the tarball method, bitbake, using the recipes, pulls down files from the online repositories and puts those files into the centralized local download directory ($DL_DIR), allowing reuse instead of re-downloading each time. The content downloaded for linux-yocto-3.0 is exactly what would be pulled from the local repository if I used a local clone of the git repository for linux-yocto-3.0. * If my assumption above is correct, if I'm not modifying the source code of the kernel (only changing config parameters), then once you've run at least one build with the tarball method, the $DL_DIR directory contains all the files you'll need to build any image with linux-yocto-3.0. So there is no need to have a local clone of the kernel repository for speeding up development. Am I right? * If I have a successful creation of a bare clone of linux-yocto-3.0.git, how could builds of Edison packages be failing? That makes me concerned about using git and successfully repeating builds of stable branches like Edison. If you set BB_GENERATE_MIRROR_TARBALLS = 1 (e.g. in local.conf) then you'll get tarballs which hold the git repositories after download. You can then reuse these (by sharing the DL_DIR or using a local mirror). Does that help with the issue you're seeing? I'm not sure it does. I don't want poky to do more work. I have my download directory, $DL_DIR, outside my build directory so I can keep it run to run, as mentioned in the comments in local.conf. I was trying to understand 2 basic questions: 1. what could be causing build failures using a freshly created bare clone yesterday vs. using the poky-edison-6.0 tarball. It would be scary if you could clone the linux-yocto-3.0 successfully one day and have it be used in a build successfully, but clone it another day and it not work. I figured that bitbake/poky pulled the same information into DL_DIR regardless of whether you pulled from the bare clone locally or straight from the on-line repository. 2. I was trying to look at items to speed up build runs. I thought that if the downloads in DL_DIR were reused if they existed, it would speed up a run. It seems to. So the question is, after the first build, the files I need for linux-yocto-3.0 are in DL_DIR regardless of whether they came from the online repository or the local bare clone. right? Jim A ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] tar ball vs. git development questions
On 2012-01-23 05:51, jfabernathy wrote:
On 01/22/2012 08:12 PM, Gary Thomas wrote:
On 2012-01-22 13:19, James Abernathy wrote:
I have used both git and the tarball methods of bitbaking projects, all of them
derivatives of the examples in the Yocto documentation. I was having issues
using the local clone of
the Yocto kernel git repository this weekend. I had successfully done that
before, but I was rebuilding the PC workstation, and getting everything setup
and tested some of the
meta-intel BSPs to make sure I had everything right. Cloning the
linux-yocto-3.0 repository was successful, but the bakes against it failed. I
made sure I had poky-extras setup
right, but I still had problems. To isolate the problem, I changed to building
with the tarballs and everything worked fine.
So that got me thinking what are the differences between the 2 methods:
* I assume that if I use the tarball method, bitbake, using the recipes, pulls
down files from the online repositories and puts those files into the
centralized local download
directory ($DL_DIR), allowing reuse instead of re-downloading each time. The
content downloaded for linux-yocto-3.0 is exactly what would be pulled from the
local repository if
I used a local clone of the git repository for linux-yocto-3.0.
* If my assumption above is correct, if I'm not modifying the source code of
the kernel (only changing config parameters), then once you've run at least one
build with the
tarball method, the $DL_DIR directory contains all the files you'll need to
build any image with linux-yocto-3.0. So there is no need to have a local clone
of the kernel
repository for speeding up development. Am I right?
* If I have a successful creation of a bare clone of linux-yocto-3.0.git, how
could builds of Edison packages be failing? That makes me concerned about using
git and successfully
repeating builds of stable branches like Edison.
If you set BB_GENERATE_MIRROR_TARBALLS = 1 (e.g. in local.conf)
then you'll get tarballs which hold the git repositories after
download. You can then reuse these (by sharing the DL_DIR or
using a local mirror). Does that help with the issue you're seeing?
I'm not sure it does. I don't want poky to do more work. I have my download
directory, $DL_DIR, outside my build directory so I can keep it run to run, as
mentioned in the comments
in local.conf. I was trying to understand 2 basic questions:
1. what could be causing build failures using a freshly created bare clone
yesterday vs. using the poky-edison-6.0 tarball. It would be scary if you could
clone the linux-yocto-3.0
successfully one day and have it be used in a build successfully, but clone it
another day and it not work. I figured that bitbake/poky pulled the same
information into DL_DIR
regardless of whether you pulled from the bare clone locally or straight from
the on-line repository.
What kind of errors? Some details might help understand what the problem is.
2. I was trying to look at items to speed up build runs. I thought that if the
downloads in DL_DIR were reused if they existed, it would speed up a run. It
seems to. So the
question is, after the first build, the files I need for linux-yocto-3.0 are in
DL_DIR regardless of whether they came from the online repository or the local
bare clone. right?
I think so, but I'm not 100% sure.
The way I have my system set up, I never download anything more
than once :-) I have a shared source repository which I point to
using MIRRORS and then all my builds are relative to that. If there
is new code, it gets downloaded and saved (as a tarball) in my sources
repository to be used by subsequent builds. To do this, I just have
these lines in distro.conf
# Provide pre-staged sources
SOURCE_MIRROR_URL ?= file://${COREBASE}/sources/
INHERIT += own-mirrors
BB_GENERATE_MIRROR_TARBALLS = 1
I don't define DL_DIR in local.conf so files only come from my mirror.
This process is so successful that I almost always run with BB_NO_NETWORK=1
which causes the fetcher to die if it can't find the file locally. If I find
that I do need to download something new, I disable this and let the fetcher
download it. I'll then have a saved tarball (either downloaded directly or
synthesized) in build_dir/downloads that I can push to my mirror.
--
Gary Thomas | Consulting for the
MLB Associates |Embedded world
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] tar ball vs. git development questions
On 12-01-23 08:10 AM, Gary Thomas wrote:
On 2012-01-23 05:51, jfabernathy wrote:
On 01/22/2012 08:12 PM, Gary Thomas wrote:
On 2012-01-22 13:19, James Abernathy wrote:
I have used both git and the tarball methods of bitbaking projects,
all of them derivatives of the examples in the Yocto documentation.
I was having issues using the local clone of
the Yocto kernel git repository this weekend. I had successfully
done that before, but I was rebuilding the PC workstation, and
getting everything setup and tested some of the
meta-intel BSPs to make sure I had everything right. Cloning the
linux-yocto-3.0 repository was successful, but the bakes against it
failed. I made sure I had poky-extras setup
right, but I still had problems. To isolate the problem, I changed
to building with the tarballs and everything worked fine.
So that got me thinking what are the differences between the 2 methods:
* I assume that if I use the tarball method, bitbake, using the
recipes, pulls down files from the online repositories and puts
those files into the centralized local download
directory ($DL_DIR), allowing reuse instead of re-downloading each
time. The content downloaded for linux-yocto-3.0 is exactly what
would be pulled from the local repository if
I used a local clone of the git repository for linux-yocto-3.0.
* If my assumption above is correct, if I'm not modifying the source
code of the kernel (only changing config parameters), then once
you've run at least one build with the
tarball method, the $DL_DIR directory contains all the files you'll
need to build any image with linux-yocto-3.0. So there is no need to
have a local clone of the kernel
repository for speeding up development. Am I right?
* If I have a successful creation of a bare clone of
linux-yocto-3.0.git, how could builds of Edison packages be failing?
That makes me concerned about using git and successfully
repeating builds of stable branches like Edison.
If you set BB_GENERATE_MIRROR_TARBALLS = 1 (e.g. in local.conf)
then you'll get tarballs which hold the git repositories after
download. You can then reuse these (by sharing the DL_DIR or
using a local mirror). Does that help with the issue you're seeing?
I'm not sure it does. I don't want poky to do more work. I have my
download directory, $DL_DIR, outside my build directory so I can keep
it run to run, as mentioned in the comments
in local.conf. I was trying to understand 2 basic questions:
1. what could be causing build failures using a freshly created bare
clone yesterday vs. using the poky-edison-6.0 tarball. It would be
scary if you could clone the linux-yocto-3.0
successfully one day and have it be used in a build successfully, but
clone it another day and it not work. I figured that bitbake/poky
pulled the same information into DL_DIR
regardless of whether you pulled from the bare clone locally or
straight from the on-line repository.
What kind of errors? Some details might help understand what the problem
is.
2. I was trying to look at items to speed up build runs. I thought
that if the downloads in DL_DIR were reused if they existed, it would
speed up a run. It seems to. So the
question is, after the first build, the files I need for
linux-yocto-3.0 are in DL_DIR regardless of whether they came from the
online repository or the local bare clone. right?
I think so, but I'm not 100% sure.
From my experience, this is true. Regardless of the SRCI_URI, the
downloads directory contains everything that poky/bitbake need to
do builds after the initial fetch has been performed. And that's
where subsequent updates are pullled.
Cheers,
Bruce
The way I have my system set up, I never download anything more
than once :-) I have a shared source repository which I point to
using MIRRORS and then all my builds are relative to that. If there
is new code, it gets downloaded and saved (as a tarball) in my sources
repository to be used by subsequent builds. To do this, I just have
these lines in distro.conf
# Provide pre-staged sources
SOURCE_MIRROR_URL ?= file://${COREBASE}/sources/
INHERIT += own-mirrors
BB_GENERATE_MIRROR_TARBALLS = 1
I don't define DL_DIR in local.conf so files only come from my mirror.
This process is so successful that I almost always run with
BB_NO_NETWORK=1
which causes the fetcher to die if it can't find the file locally. If I
find
that I do need to download something new, I disable this and let the
fetcher
download it. I'll then have a saved tarball (either downloaded directly or
synthesized) in build_dir/downloads that I can push to my mirror.
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
