Hi Andrew, I'm getting an error when I apply your patch. Do you now how can I work around this?
user@pc:~/yocto/pyro/poky$ git fetch user@pc:~/yocto/pyro/poky$ git reset user@pc:~/yocto/pyro/poky$ git clean -fd user@pc:~/yocto/pyro/poky$ git reset --hard HEAD is now at b51b4f5 gawk: Enable native building user@pc:~/yocto/pyro/poky$ git branch * pyro user@pc:~/yocto/pyro/poky$ git apply ../0001-Revert-w-modifications-bitbake-fetch2-perforce-Rewor.patch ../0001-Revert-w-modifications-bitbake-fetch2-perforce-Rewor.patch:28: trailing whitespace. 'DBUS_SESSION_BUS_ADDRESS'] ../0001-Revert-w-modifications-bitbake-fetch2-perforce-Rewor.patch:41: trailing whitespace. BitBake 'Fetch' implementations ../0001-Revert-w-modifications-bitbake-fetch2-perforce-Rewor.patch:42: trailing whitespace. ../0001-Revert-w-modifications-bitbake-fetch2-perforce-Rewor.patch:43: trailing whitespace. Classes for obtaining upstream sources for the ../0001-Revert-w-modifications-bitbake-fetch2-perforce-Rewor.patch:44: trailing whitespace. BitBake build tools. error: patch failed: bitbake/lib/bb/fetch2/__init__.py:817 error: bitbake/lib/bb/fetch2/__init__.py: patch does not apply error: patch failed: bitbake/lib/bb/fetch2/perforce.py:1 error: bitbake/lib/bb/fetch2/perforce.py: patch does not apply Cheers, Katu 2017-09-14 17:17 GMT+01:00 Andrew Bradford <and...@bradfordembedded.com>: > Hi Katu, > > Sorry for my delay in responding. > > On 09/14 11:13, Katu Txakur wrote: > > Have you had time to look at this? I tried to go back to the old perforce > > fetcher but I got a license error and I couldn't work around it. > > On the pyro branch of poky, if you revert these two commits, in this > order: > > 35081f55185a8a9804c21b16cd4600ba70646a10 "bitbake: bitbake-user-manual: > Addeds support for the Perforce Fetcher" > 40f3099e8ec5c6f3a8b7f3d0e90f1c65c388ee77 "base.bbclass: p4 fetcher > supports srcrev" > > And then apply the patch attached, this should bring the pyro branch of > poky back to how the old perforce fetcher worked. I have tested having > two different depot paths in a recipe SRC_URI and they both are fetched > how I believe you want them to be, next to each other in WORKDIR. > > Just be aware, I believe this loses the ability to use > SRC_REV="${AUTOREV}" to automatically find the newest changelist which > my previous changes introduced. > > The format for SRC_URI that I tested was like: > > SRC_URI = "p4://username:password:server.hostname.example.com:1666@depot > /path/to/directory/...;cset=178374" > > You should also be able to specify P4PORT to set the host and port > number but you may then lose the ability to have the username and > password (although I haven't tested this and don't remember how it used > to work). You should also be able to specify the P4DATE variable in > your recipe to apply to all p4 fetches instead of using the "cset=" > parameter, and instead of using the "cset=" param you should also be > able to specify a "revision=" for a single file or a "label=" for a > label. I've only tested using the "cset=" way as the others don't > easily apply to how my team internally uses perforce, sorry. > > I think I understand how I could make the current perforce fetcher > (without the above reverts or attached patch) do the multi-directory > fetching that you want, but I don't personally want to do that in my > workflow so I'm not going to spend a bunch of time implementing it now. > But if you do implement it, I'd be happy to test patches for you. > > Thanks, > Andrew > > > 2017-08-31 18:54 GMT+01:00 Andrew Bradford <and...@bradfordembedded.com > >: > > > > > Hi Katu, > > > > > > On 08/28 17:43, Katu Txakur wrote: > > > > Thanks for looking at this. See my recipe below. I've left the bits > > > related > > > > to systemd service but I don't think they matter for this. I'm using > an > > > old > > > > implementation of Perforce (2010) in case this matters. I've tried > going > > > > back to the old perforce.py fetcher but I get a license error... do > you > > > > know if it would be easy to revert to the old version in my bitbake > > > folder > > > > until we make this work? > > > > > > Sorry, I've been swamped this week and haven't been able to look into > > > this yet. This coming weekend is a holiday weekend in the USA, too. > > > But I plan to look at this early next week, hope that's OK. > > > > > > We should be able to create a patch series to revert my changes so you > > > can go back to the old perforce fetcher. It might also be worth > > > investigating how to take the current perforce fetcher and enable some > > > of the use cases that you have (but we can do this after we > successfully > > > revert). > > > > > > I'll try to send a patch to you next week for the reverting. > > > Thanks, > > > Andrew > > > > > > > DESCRIPTION = "Dummy recipe to fetch from Perforce" > > > > SECTION = "PerforceRecipes" > > > > LICENSE = "CLOSED" > > > > PR = "r0" > > > > > > > > inherit cmake > > > > DEPENDS = "boost alsa-tools" > > > > > > > > P4USER = "myuser" > > > > P4PASSWD = "" > > > > P4PORT = "192.168.1.55:1666" > > > > FETCHCMD_p4 = "/usr/local/bin/p4" > > > > > > > > SRC_URI = " \ > > > > p4://${P4USER}:${P4PASSWD}@myrepo/myfolder/...; \ > > > > file://perforce-recipe.service \ > > > > " > > > > SRCREV = "${AUTOREV}" > > > > S = "${WORKDIR}/p4" > > > > > > > > do_install_append () { > > > > install -d ${D}${systemd_unitdir}/system > > > > install -m 0644 ../perforce-recipe.service > > > ${D}${systemd_unitdir}/system > > > > } > > > > > > > > NATIVE_SYSTEMD_SUPPORT = "1" > > > > SYSTEMD_PACKAGES = "perforce-recipe" > > > > SYSTEMD_SERVICE_${PN} = "perforce-recipe.service" > > > > > > > > SYSTEMD_AUTO_ENABLE = "disable" > > > > > > > > FILES_${PN} = "${systemd_unitdir} ${bindir}" > > > > > > > > > > > > 2017-08-28 15:27 GMT+01:00 Andrew Bradford < > and...@bradfordembedded.com > > > >: > > > > > > > > > Hi Katu, > > > > > > > > > > On 08/28 08:56, Katu Txakur wrote: > > > > > > 2017-08-11 13:52 GMT+01:00 Andrew Bradford < > > > and...@bradfordembedded.com > > > > > >: > > > > > > On 08/11 11:34, Katu Txakur wrote: > > > > > > > > 2017-08-07 12:17 GMT+01:00 Andrew Bradford < > > > > > and...@bradfordembedded.com > > > > > > > >: > > > > > > > > > On 08/02 10:28, Paul Eggleton wrote: > > > > > > > > > > On Wednesday, 2 August 2017 9:40:10 AM CEST Katu Txakur > > > wrote: > > > > > > > > > > > I'm still having problems fetching from Perforce. Is > there > > > any > > > > > > > > > > > documentation based on the latest implementation of the > > > > > > > > > > > poky/bitbake/lib/bb/fetch2/perforce.py file? > > > > > > > > > > > > > > > > > > There's some documentation now in the official Yocto > Project > > > > > > > > > documentation location for bitbake fetchers [1]. Prior to > my > > > > > changes > > > > > > > > > there was very little if any documentation, afaik. > > > > > > > > > > > > > > > > > > [1]: http://www.yoctoproject.org/docs/2.3.1/bitbake-user- > > > > > > > > > manual/bitbake-user-manual.html#perforce-fetcher > > > > > > > > > > > > > > > > > > > > 2017-07-25 10:05 GMT+01:00 Katu Txakur < > > > katutxaku...@gmail.com > > > > > >: > > > > > > > > > > > > I'm upgrading a recipe that fetches the source code > from > > > > > > > Perforce. > > > > > > > > > > > > > > > > > > > > > > > > The old recipe was: > > > > > > > > > > > > > > > > > > > > > > > > SRC_URI = " \ > > > > > > > > > > > > p4://${P4USER}:${P4PASSWD}:${ > P4HOST}:${P4PORT}@Depot > > > /path/ > > > > > pe > > > > > > > > > > > > rforce/...;module=local/path/relativeto/p4;label=${ > > > > > P4CHANGELIST} > > > > > > > \ > > > > > > > > > > > > " > > > > > > > > > > > > > > > > > > > > > > > > With the new version of /lib/bb/fetch2/perforce.py, I > > > had to > > > > > set > > > > > > > > > P4PORT > > > > > > > > > > > > outside SRC_URI, leaving the recipe with: > > > > > > > > > > > > > > > > > > > > > > > > SRC_URI = " \ > > > > > > > > > > > > p4://${P4USER}:${P4PASSWD}@ > Depot/path/perforce/...; > > > module= > > > > > > > > > > > > local/path/relativeto/p4;label=${P4CHANGELIST} \ > > > > > > > > > > > > " > > > > > > > > > > > > > > > > > > > > > > > > The Perforce fetcher kind of works, but it seems to > be > > > > > ignoring > > > > > > > the > > > > > > > > > > > > "module" and the "label" attributes. After reading > the > > > Python > > > > > > > script > > > > > > > > > I can > > > > > > > > > > > > see that after the "@", only the substring before the > > > first > > > > > ";" > > > > > > > is > > > > > > > > > used. > > > > > > > > > > > > > > > > > > Sorry for not making the label change more clear, but you > > > should be > > > > > > > able > > > > > > > > > to simply set SRCREV="labelname" and have the > functionality you > > > > > desire. > > > > > > > > > However, we don't use labels very often internally, so the > use > > > of > > > > > > > labels > > > > > > > > > in SRCREV isn't tested that often by me. > > > > > > > > > > > > > > > > > > > > > > > > > I don't use labels that much either, however, I used to be > able > > > to > > > > > write > > > > > > > > the changeslist number as a label and it would > > > > > > > > fetch the code for that changelist. This doesn't work using > > > SRCREV = > > > > > > > > "changelistnumber" > > > > > > > > > > > > > > This does work for me using oe-core morty branch and bitbake > > > > > > > 1.32 which is the versions where these changes were first > applied > > > and > > > > > > > where I did my testing. > > > > > > > > > > > > > > Like I have a recipe which I put: > > > > > > > > > > > > > > SRCREV = "184127" > > > > > > > > > > > > > > The p4 fetcher will go fetch changelist 184127 of the SRC_URI > path > > > in > > > > > > > perforce. > > > > > > > > > > > > > > Or if I put: > > > > > > > > > > > > > > SRCREV = "${AUTOREV}" > > > > > > > > > > > > > > Then it'll fetch the HEAD/tip of the SRC_URI path in perforce. > > > > > > > > > > > > > > Is this not working the same way with poky now? > > > > > > > > > > > > > > > > > > > I'm using yocto pyro with bitbake 1.34.0. When I first fetch the > > > code for > > > > > > my recipe, it gets the latest changelist. > > > > > > I submit a change and the next time, the recipe thinks there > nothing > > > new > > > > > to > > > > > > fetch. Also, using SRCREV = "184127" > > > > > > doesn't work for me, it always gets the latest changelist. > > > > > > > > > > Can you post the recipe where this is failing for you? > > > > > > > > > > I could then try to recreate this issue in a sandbox depot we have > in > > > > > our perforce server and see what's going on. > > > > > > > > > > > > > > > > > > > > > This change to using labels, changelist numbers, and > allowing > > > the > > > > > use > > > > > > > of > > > > > > > > > AUTOREV was made so that the perforce fetcher would act > more > > > like > > > > > the > > > > > > > > > other source code control system fetchers. Prior to my > > > changes to > > > > > the > > > > > > > > > p4 fetcher, the way that recipes using p4 had to be written > > > seemed > > > > > > > > > rather awkward to me. The way it is now isn't perfect, but > > > it's > > > > > closer > > > > > > > > > to how the other fetchers act, I think (and hope). > > > > > > > > > > > > > > > > > > > > > > > > > > > > Is there a way to set module and label/changelist? I > have > > > > > several > > > > > > > > > folders > > > > > > > > > > > > per recipe that I need to map to different > subfolders and > > > > > with > > > > > > > the > > > > > > > > > current > > > > > > > > > > > > script some of the folders don't get fetched. > > > > > > > > > > > > > > > > > > I'm not sure I understand enough about how you use > perforce and > > > > > what > > > > > > > you > > > > > > > > > mean by mapping different subfolders. Can you explain > this in > > > a > > > > > more > > > > > > > > > expanded way to me? Maybe with some examples? > > > > > > > > > > > > > > > > > > > > > > > > > What I mean by this is taking folders in different locations > in > > > > > Perforce > > > > > > > > and fetch them to a custom folder structure inside the > > > {WORKDIR}/p4 > > > > > > > folder. > > > > > > > > This is an example of how I was doing this with the old > version > > > of > > > > > the > > > > > > > > fetcher: > > > > > > > > > > > > > > > > #leave blank, "", for latest revision > > > > > > > > P4CHANGELIST = "${PR}" > > > > > > > > S = "${WORKDIR}" > > > > > > > > SRC_URI = " \ > > > > > > > > p4://${P4USER}:${P4PASSWD}:${P4HOST}:${P4PORT}@Depot/project > > > > > > > 1/folder1/...;module=subfolderunderp4/src/;label=${ > P4CHANGELIST} > > > > > > > > \ > > > > > > > > p4://${P4USER}:${P4PASSWD}:${P4HOST}:${P4PORT}@Depot/project > > > > > > > 2/folder3/...;module=subfolderunderp4/pkg/;label=${ > P4CHANGELIST} > > > > > > > > \ > > > > > > > > " > > > > > > > > > > > > > > > > Where folder1 contains the src files and folder3 contains > the pkg > > > > > > > > information. These need to be in an specific folder structure > > > for the > > > > > > > > recipe to work. > > > > > > > > I had some issue when adding more than one line because it > was > > > > > trying to > > > > > > > > copy everything to the same p4 folder. > > > > > > > > > > > > > > Ah, yeah... Sorry, that is a bit of a shortcoming of my > changes. > > > The > > > > > > > reason I made this change is because putting the given > SRC_URI's > > > > > > > repository directly into a directory like ${WORKDIR}/p4/ is how > > > some of > > > > > > > the other fetchers operate and it makes the automatic tasks > just > > > work > > > > > > > correctly. This is how the git fetcher does things so I tried > to > > > > > mimmic > > > > > > > that but maybe I missed some nuance in how the git fetcher > deals > > > with > > > > > > > more than one SRC_URI line being a git repo. I hadn't realized > > > that > > > > > > > there was users of the p4 fetcher who did things like you are > > > doing, > > > > > > > sorry. > > > > > > > > > > > > > > I think it should be reasonably easy to make a patch to the p4 > > > fetcher > > > > > > > which will put each SRC_URI line which uses the p4 fetcher into > > > its own > > > > > > > directory within the ${WORKDIR}/p4/ directory such that it > supports > > > > > your > > > > > > > style of fetching. This then will likely require your recipe > to > > > know > > > > > > > that this is the directory structure and properly change into > > > > > > > directories as needed, but it sounds like you already > comprehend > > > that. > > > > > > > And for users of the p4 fetcher who only use one SRC_URI line, > I > > > > > believe > > > > > > > they would just need to set the S variable correctly to the > > > directory > > > > > > > name within ${WORKDIR}/p4/ to get the current operation. > > > > > > > > > > > > I have been looking at this and it wasn't straight forward for > me... > > > I > > > > > > don't have much experience with this. I will continue trying but > I > > > might > > > > > > need some help. > > > > > > It's not only not being able to fetch more than one SRC_URI. We > are > > > also > > > > > > loosing the flexibility of fetching different changelists for > each > > > line. > > > > > > I'm not sure if this can be solved in this new fetcher whilst > > > keeping git > > > > > > and p4 "similar" as git doesn't work this way afaik. > > > > > > > > > > It sounds like you really just want to revert to how the p4 fetcher > > > used > > > > > to work. My changes were because I wanted the p4 fetcher to work > more > > > > > like the git and svn fetchers. But that doesn't sound like what > you > > > > > want. > > > > > > > > > > My p4 fetcher changes which are impacting you on poky's pyro are > these: > > > > > > > > > > 35081f55185a8a9804c21b16cd4600ba70646a10 bitbake: > bitbake-user-manual: > > > > > Addeds support for the Perforce Fetcher > > > > > 40f3099e8ec5c6f3a8b7f3d0e90f1c65c388ee77 base.bbclass: p4 fetcher > > > > > supports srcrev > > > > > 26447a0d0bf5ceaac382a78c1dbd00807dbab706 bitbake: fetch2/perforce: > > > Rework > > > > > to support SRCREV and P4CONFIG > > > > > > > > > > But that last one will run into conflicts if you just try to > revert it > > > > > directly due to having other commits since then which git can't > easily > > > > > figure out, but if you revert all of these commits too, then you > may > > > > > have to redo some of the changes they made which are related to > general > > > > > things and are not p4 specific. Commits you'll want to look at > are: > > > > > > > > > > b16192c93834d0a6530169557aa34122e1417bcf bitbake: fetch2: don't > use > > > > > deprecated bb.data APIs > > > > > 6c611d697f9c03867c938cba1b481f38eebed8bf bitbake: fetch2: Rename > > > > > "setup_revisons" to "setup_revisions" > > > > > 38438b6cf42fb7ad45b9a901f57913af7e7591a3 bitbake: fetch2: obey > > > > > BB_ALLOWED_NETWORKS when checking network access > > > > > 1fce7ecbbb004a5ad82da3eef79cfd52b276708d bitbake: bitbake: remove > True > > > > > option to getVar calls > > > > > ab09541d5517da9b1a23923ea8f5c26ddf745084 bitbake: fetch2: preserve > > > > > current working directory > > > > > 26447a0d0bf5ceaac382a78c1dbd00807dbab706 bitbake: fetch2/perforce: > > > Rework > > > > > to support SRCREV and P4CONFIG > > > > > > > > > > I haven't actually reviewed any of these commits after the > "preserve > > > > > current working directory" one as I'm still doing my day-to-day > work on > > > > > the morty branch of oe-core and bitbake 1.32. But possibly one of > > > these > > > > > commits are unintentionally impacting your ability to specify > SRCREV > > > and > > > > > that's worth checking into. > > > > > > > > > > Thanks, > > > > > Andrew > > > > > > > > >
-- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto