Re: [yocto] SRCREV how is it supposed to work?
On 2013-11-05 11:10, Robert Calhoun wrote:
> -Original Message-
> From: Paul Eggleton
>> AFAIK, there are two recommended values for SRCREV assuming you are
>> fetching
> >from an SCM at all:
>> A) A specific revision (SHA1 hash when fetching from git)
>>
>> or
>>
>> B) "${AUTOREV}" if you want to always build the latest version available
>> at
>> time of building. If you want to build the latest version from a branch,
>> specify it in branch= in the SRC_URI entry.
>>
>> Anything else isn't really a good idea. Sometimes I wonder if we ought to
>> just
>> tighten this up so that only settings that make sense can be set.
> The current behavior is a little non-intuitive, since using SRCREV =
> "${AUTOREV}"
> alone will not force the package to be rebuilt when SRCREV changes. Unless
> I am
> mistaken, $PV needs to be modified also. But modifying $PV causes its own
> headaches with patching, so I've ended up using recipes based on the model
> below.
>
> Another challenge is that bitbake's fetch2 is not very well documented. I
> don't
> think the "user" and "pswd" fields for the svn fetcher are documented
> anywhere
> outside of the source code.
>
> I'd love to help address this, but I'm not sure where I should submit
> updated documentation. Is this the right place?
>
> git://git.yoctoproject.org/yocto-docs
>
>
> Hans, below is a model recipe for building current head-of-line from a
> subversion repository:
A good example indeed. I will see what I can make out of it in our own recipes.
I also realized the caveats attached to finding the patch dir etc. when
auto-incrementing the version.
This will certainly help.
Thanks.
Hans
> -Rob Calhoun
> SST Inc
>
>
>
>
> ##
>
> DESCRIPTION = "example_1.0.bb, autorevving checkout example"
>
> # This says look for LICENSE in the root of the directory being checked
> out. Fix
> # license filename and md5sum as required.
> LIC_FILES_CHKSUM = "file://LICENSE;md5=abc123"
>
> # this is the rev of your recipe. Bumping it will toss the cache and redo
> everything
> PR = "r1"
>
> # pick up latest svn rev for this module. Note this a deferred evaluation!
> SRCREV = "${AUTOREV}"
>
> # ${PV} is pulled from the recipe filename, e.g. helloworld_2.7.bb ->
> ${PV} expands to "2.7".
> # We use immediate evaluation to define a new var PVBASE holding the
> original value of ${PV}.
> PVBASE := "${PV}"
>
> # We need to tell bitbake about our current directory structure or we
> won't be able to find patches after we mess with ${PV}
> FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PVBASE}:"
>
> # Then redefine PV to tack the svn rev onto the base version. This is
> evaluated at fetch time.
> # Then the package will get rebuilt any time the relevant part of the
> source tree changes.
> PV = "${PVBASE}.${SRCPV}"
>
> # Now we format the source code URI.
> # There is nothing special about "module"; it is just the final directory
> of the svn checkout.
> # SRC_URI below is equivalent to the svn command:
> # "svn checkout --username=poky --password=xyzzy
> https://svn.example.com/repo/trunk/example";
> #
> SRC_URI=
> "svn://svn.example.com/repo/trunk;module=example;protocol=https;user=poky;p
> swd=xyzzy"
>
> # build will fail without this; it specifies where in the workdir to find
> the source. The
> # name must be the same as the "module" name in SRC_URI.
> S = "${WORKDIR}/example"
>
> # BELOW NOT PART OF RECIPE; IT SHOWS WHAT THE WORK DIR LOOKS LIKE WITH
> THIS RECIPE.
> # By setting PV, the cache is invalidated and new code checked out each
> time the
> # relevant part oF the svn repo gets updated because I've made the svn
> revision look
> # like a package version number to bitbake.
> #
> # Here is a directory listing of the work dir:
> #
> #
> poky@build:/poky/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/example$
> ls -l
> #drwxrwxr-x 12 poky poky 4096 Oct 30 10:17 1.0.57400-r1
> #drwxrwxr-x 12 poky poky 4096 Oct 30 11:52 1.0.57401-r1
> #drwxrwxr-x 12 poky poky 4096 Oct 31 12:37 1.0.57408-r1
> #drwxrwxr-x 12 poky poky 4096 Nov 1 07:56 1.0.57412-r1
>
>
>
>
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] SRCREV how is it supposed to work?
-Original Message-
From: Paul Eggleton
>
>AFAIK, there are two recommended values for SRCREV assuming you are
>fetching
>from an SCM at all:
>
>A) A specific revision (SHA1 hash when fetching from git)
>
>or
>
>B) "${AUTOREV}" if you want to always build the latest version available
>at
>time of building. If you want to build the latest version from a branch,
>specify it in branch= in the SRC_URI entry.
>
>Anything else isn't really a good idea. Sometimes I wonder if we ought to
>just
>tighten this up so that only settings that make sense can be set.
The current behavior is a little non-intuitive, since using SRCREV =
"${AUTOREV}"
alone will not force the package to be rebuilt when SRCREV changes. Unless
I am
mistaken, $PV needs to be modified also. But modifying $PV causes its own
headaches with patching, so I've ended up using recipes based on the model
below.
Another challenge is that bitbake's fetch2 is not very well documented. I
don't
think the "user" and "pswd" fields for the svn fetcher are documented
anywhere
outside of the source code.
I'd love to help address this, but I'm not sure where I should submit
updated documentation. Is this the right place?
git://git.yoctoproject.org/yocto-docs
Hans, below is a model recipe for building current head-of-line from a
subversion repository:
-Rob Calhoun
SST Inc
##
DESCRIPTION = "example_1.0.bb, autorevving checkout example"
# This says look for LICENSE in the root of the directory being checked
out. Fix
# license filename and md5sum as required.
LIC_FILES_CHKSUM = "file://LICENSE;md5=abc123"
# this is the rev of your recipe. Bumping it will toss the cache and redo
everything
PR = "r1"
# pick up latest svn rev for this module. Note this a deferred evaluation!
SRCREV = "${AUTOREV}"
# ${PV} is pulled from the recipe filename, e.g. helloworld_2.7.bb ->
${PV} expands to "2.7".
# We use immediate evaluation to define a new var PVBASE holding the
original value of ${PV}.
PVBASE := "${PV}"
# We need to tell bitbake about our current directory structure or we
won't be able to find patches after we mess with ${PV}
FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}-${PVBASE}:"
# Then redefine PV to tack the svn rev onto the base version. This is
evaluated at fetch time.
# Then the package will get rebuilt any time the relevant part of the
source tree changes.
PV = "${PVBASE}.${SRCPV}"
# Now we format the source code URI.
# There is nothing special about "module"; it is just the final directory
of the svn checkout.
# SRC_URI below is equivalent to the svn command:
# "svn checkout --username=poky --password=xyzzy
https://svn.example.com/repo/trunk/example";
#
SRC_URI=
"svn://svn.example.com/repo/trunk;module=example;protocol=https;user=poky;p
swd=xyzzy"
# build will fail without this; it specifies where in the workdir to find
the source. The
# name must be the same as the "module" name in SRC_URI.
S = "${WORKDIR}/example"
# BELOW NOT PART OF RECIPE; IT SHOWS WHAT THE WORK DIR LOOKS LIKE WITH
THIS RECIPE.
# By setting PV, the cache is invalidated and new code checked out each
time the
# relevant part oF the svn repo gets updated because I've made the svn
revision look
# like a package version number to bitbake.
#
# Here is a directory listing of the work dir:
#
#
poky@build:/poky/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/example$
ls -l
#drwxrwxr-x 12 poky poky 4096 Oct 30 10:17 1.0.57400-r1
#drwxrwxr-x 12 poky poky 4096 Oct 30 11:52 1.0.57401-r1
#drwxrwxr-x 12 poky poky 4096 Oct 31 12:37 1.0.57408-r1
#drwxrwxr-x 12 poky poky 4096 Nov 1 07:56 1.0.57412-r1
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] SRCREV how is it supposed to work?
On Tuesday 05 November 2013 10:25:24 Hans Beckérus wrote:
> I have seen recipes that defines PV = "xyz+git${SRCPV}" and then
> SRCREV to a specific tag instead of ${AUTOREV}.
> Is that not contradictory? Or is it simply that the author of that
> recipe wished to have a git tag automatically added to the version
> string?
> Are there any specific cases when that approach is to prefere?
AFAIK, there are two recommended values for SRCREV assuming you are fetching
from an SCM at all:
A) A specific revision (SHA1 hash when fetching from git)
or
B) "${AUTOREV}" if you want to always build the latest version available at
time of building. If you want to build the latest version from a branch,
specify it in branch= in the SRC_URI entry.
Anything else isn't really a good idea. Sometimes I wonder if we ought to just
tighten this up so that only settings that make sense can be set.
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] SRCREV how is it supposed to work?
On Tue, Oct 29, 2013 at 3:20 PM, Hans Beckérus wrote:
> On Tue, Oct 29, 2013 at 2:42 PM, Martin Jansa wrote:
>> On Tue, Oct 29, 2013 at 02:27:30PM +0100, Hans Beckérus wrote:
>>> On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa
>>> wrote:
>>> > On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
>>> >> Hi. I am wondering if we are using SRCREV wrong somehow.
>>> >> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
>>> >> to the remote should be automatically detected and downloaded/fetched?
>>> >> I can no see that this is actually what happens. Any changes made to
>>> >> the remote still need to be manually fetched or indirectly by stepping
>>> >> the recipe revision.
>>> >> Are we using SRCREV wrong or is this actually the way it is supposed
>>> >> to work? Also, is there some way to force a download to me made every
>>> >> single time by a recipe,
>>> >> irrespective of if the remote changed or not?
>>> >
>>> > It's supposed to run git ls-remote while parsing to get latest revision
>>> > in remote repo and then rebuild the package because of SRCPV change in
>>> > PV, are you using something like:
>>> >
>>> > PV = "1.0+git${SRCPV}"
>>> >
>>> > ?
>>> >
>>> Nope. I did not know we had to use PV. Sounds like we need to in the
>>> general case. But in this case,
>>
>> You need to reference it somewhere, if you were using multiple git repos
>> in SRC_URI you probably need to define SRCREV_foo = "${AUTOREV}" for all
>> of them (where foo is from name=foo param for each repo).
>>
> Sure.
>
>>> we actually do not have versions on the package itself since it is
>>> simply a container for several other binary packages merged into one
>>> binary file.
>>> So our "package" is downloading packages from several git repos,
>>
>> "package" -> "recipe"
>>
> Yea, yea! :)
>
>>> stored in different folders using 'destsuffix'.
>>> Would it be ok to simply set PV = "${SRCPV}" ? I guess this will
>>
>> SRCPV is sortable and consistent only with shared persistent PR server is
>> used
>> across all builders - if that's not the case it's safer to prefix SRCPV
>> with some manually maintained version.
>>
> No problem. That is doable.
>
>>> result in a new target folder for each changed remote? And will this
>>> work when referring to several git repos in SRC_URI?
>>
>> You'll need to define SRCREV_FORMAT and name parameter for each repo in
>> SRC_URI.
>>
> Now I am not following you :( name I understand, but what is SRCREV_FORMAT?
>
>>> Since changes are expected to happen frequently some sort of
>>> garbage-collection function is needed to get rid of all the obsolete
>>> package trees. IIRC was there not an option for this somewhere?
>>
>> Maybe you're talking about rm_work, but I'm not sure what you mean by
>> package trees.
>>
> Maybe it was rm_work, by "package trees" I mean whatever is unpacked
> from the fetch.
>
I have seen recipes that defines PV = "xyz+git${SRCPV}" and then
SRCREV to a specific tag instead of ${AUTOREV}.
Is that not contradictory? Or is it simply that the author of that
recipe wished to have a git tag automatically added to the version
string?
Are there any specific cases when that approach is to prefere?
Thanks.
Hans
>> --
>> Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] SRCREV how is it supposed to work?
On Tue, Oct 29, 2013 at 2:42 PM, Martin Jansa wrote:
> On Tue, Oct 29, 2013 at 02:27:30PM +0100, Hans Beckérus wrote:
>> On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa
>> wrote:
>> > On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
>> >> Hi. I am wondering if we are using SRCREV wrong somehow.
>> >> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
>> >> to the remote should be automatically detected and downloaded/fetched?
>> >> I can no see that this is actually what happens. Any changes made to
>> >> the remote still need to be manually fetched or indirectly by stepping
>> >> the recipe revision.
>> >> Are we using SRCREV wrong or is this actually the way it is supposed
>> >> to work? Also, is there some way to force a download to me made every
>> >> single time by a recipe,
>> >> irrespective of if the remote changed or not?
>> >
>> > It's supposed to run git ls-remote while parsing to get latest revision
>> > in remote repo and then rebuild the package because of SRCPV change in
>> > PV, are you using something like:
>> >
>> > PV = "1.0+git${SRCPV}"
>> >
>> > ?
>> >
>> Nope. I did not know we had to use PV. Sounds like we need to in the
>> general case. But in this case,
>
> You need to reference it somewhere, if you were using multiple git repos
> in SRC_URI you probably need to define SRCREV_foo = "${AUTOREV}" for all
> of them (where foo is from name=foo param for each repo).
>
Sure.
>> we actually do not have versions on the package itself since it is
>> simply a container for several other binary packages merged into one
>> binary file.
>> So our "package" is downloading packages from several git repos,
>
> "package" -> "recipe"
>
Yea, yea! :)
>> stored in different folders using 'destsuffix'.
>> Would it be ok to simply set PV = "${SRCPV}" ? I guess this will
>
> SRCPV is sortable and consistent only with shared persistent PR server is used
> across all builders - if that's not the case it's safer to prefix SRCPV
> with some manually maintained version.
>
No problem. That is doable.
>> result in a new target folder for each changed remote? And will this
>> work when referring to several git repos in SRC_URI?
>
> You'll need to define SRCREV_FORMAT and name parameter for each repo in
> SRC_URI.
>
Now I am not following you :( name I understand, but what is SRCREV_FORMAT?
>> Since changes are expected to happen frequently some sort of
>> garbage-collection function is needed to get rid of all the obsolete
>> package trees. IIRC was there not an option for this somewhere?
>
> Maybe you're talking about rm_work, but I'm not sure what you mean by
> package trees.
>
Maybe it was rm_work, by "package trees" I mean whatever is unpacked
from the fetch.
> --
> Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] SRCREV how is it supposed to work?
On Tue, Oct 29, 2013 at 02:27:30PM +0100, Hans Beckérus wrote:
> On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa wrote:
> > On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
> >> Hi. I am wondering if we are using SRCREV wrong somehow.
> >> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
> >> to the remote should be automatically detected and downloaded/fetched?
> >> I can no see that this is actually what happens. Any changes made to
> >> the remote still need to be manually fetched or indirectly by stepping
> >> the recipe revision.
> >> Are we using SRCREV wrong or is this actually the way it is supposed
> >> to work? Also, is there some way to force a download to me made every
> >> single time by a recipe,
> >> irrespective of if the remote changed or not?
> >
> > It's supposed to run git ls-remote while parsing to get latest revision
> > in remote repo and then rebuild the package because of SRCPV change in
> > PV, are you using something like:
> >
> > PV = "1.0+git${SRCPV}"
> >
> > ?
> >
> Nope. I did not know we had to use PV. Sounds like we need to in the
> general case. But in this case,
You need to reference it somewhere, if you were using multiple git repos
in SRC_URI you probably need to define SRCREV_foo = "${AUTOREV}" for all
of them (where foo is from name=foo param for each repo).
> we actually do not have versions on the package itself since it is
> simply a container for several other binary packages merged into one
> binary file.
> So our "package" is downloading packages from several git repos,
"package" -> "recipe"
> stored in different folders using 'destsuffix'.
> Would it be ok to simply set PV = "${SRCPV}" ? I guess this will
SRCPV is sortable and consistent only with shared persistent PR server is used
across all builders - if that's not the case it's safer to prefix SRCPV
with some manually maintained version.
> result in a new target folder for each changed remote? And will this
> work when referring to several git repos in SRC_URI?
You'll need to define SRCREV_FORMAT and name parameter for each repo in
SRC_URI.
> Since changes are expected to happen frequently some sort of
> garbage-collection function is needed to get rid of all the obsolete
> package trees. IIRC was there not an option for this somewhere?
Maybe you're talking about rm_work, but I'm not sure what you mean by
package trees.
--
Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
signature.asc
Description: Digital signature
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] SRCREV how is it supposed to work?
On Tue, Oct 29, 2013 at 12:00 PM, Martin Jansa wrote:
> On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
>> Hi. I am wondering if we are using SRCREV wrong somehow.
>> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
>> to the remote should be automatically detected and downloaded/fetched?
>> I can no see that this is actually what happens. Any changes made to
>> the remote still need to be manually fetched or indirectly by stepping
>> the recipe revision.
>> Are we using SRCREV wrong or is this actually the way it is supposed
>> to work? Also, is there some way to force a download to me made every
>> single time by a recipe,
>> irrespective of if the remote changed or not?
>
> It's supposed to run git ls-remote while parsing to get latest revision
> in remote repo and then rebuild the package because of SRCPV change in
> PV, are you using something like:
>
> PV = "1.0+git${SRCPV}"
>
> ?
>
Nope. I did not know we had to use PV. Sounds like we need to in the
general case. But in this case,
we actually do not have versions on the package itself since it is
simply a container for several other binary packages merged into one
binary file.
So our "package" is downloading packages from several git repos,
stored in different folders using 'destsuffix'.
Would it be ok to simply set PV = "${SRCPV}" ? I guess this will
result in a new target folder for each changed remote? And will this
work when referring to several git repos in SRC_URI?
Since changes are expected to happen frequently some sort of
garbage-collection function is needed to get rid of all the obsolete
package trees. IIRC was there not an option for this somewhere?
>>
>> Thanks.
>> Hans
>> ___
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>
> --
> Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] SRCREV how is it supposed to work?
On Tue, Oct 29, 2013 at 12:46:18PM +0100, Hans Beckérus wrote:
> Hi. I am wondering if we are using SRCREV wrong somehow.
> Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
> to the remote should be automatically detected and downloaded/fetched?
> I can no see that this is actually what happens. Any changes made to
> the remote still need to be manually fetched or indirectly by stepping
> the recipe revision.
> Are we using SRCREV wrong or is this actually the way it is supposed
> to work? Also, is there some way to force a download to me made every
> single time by a recipe,
> irrespective of if the remote changed or not?
It's supposed to run git ls-remote while parsing to get latest revision
in remote repo and then rebuild the package because of SRCPV change in
PV, are you using something like:
PV = "1.0+git${SRCPV}"
?
>
> Thanks.
> Hans
> ___
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
--
Martin 'JaMa' Jansa jabber: martin.ja...@gmail.com
signature.asc
Description: Digital signature
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[yocto] SRCREV how is it supposed to work?
Hi. I am wondering if we are using SRCREV wrong somehow.
Is it expected that if we use SRCREV = "${AUTOREV}", that any changes
to the remote should be automatically detected and downloaded/fetched?
I can no see that this is actually what happens. Any changes made to
the remote still need to be manually fetched or indirectly by stepping
the recipe revision.
Are we using SRCREV wrong or is this actually the way it is supposed
to work? Also, is there some way to force a download to me made every
single time by a recipe,
irrespective of if the remote changed or not?
Thanks.
Hans
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
