Re: Use of exists() in a Makefile
Le 10/06/11 17:05, David Cantrell a écrit :
On 10/06/2011 09:55 AM, Stuart Henderson wrote:
On 2011-10-04, David Cantrelldavid.l.cantr...@gmail.com wrote:
My suggestion was to add support to the ports system infrastructure to
allow people an easy way to locally package up stuff from projects that
do not release tarballs.
You could add something like this to your Makefile
create-tarball:
cd ${FULLDISTDIR} \
svn export http://foo ${DISTNAME} \
tar czf ${DISTNAME}${EXTRACT_SUFX} ${DISTNAME}
FETCH_MANUALLY= Use: make create-tarball
Now that's something that could work in this particular case and it
still fits nicely within the ports infrastructure.
Thanks for the suggestion!
Also, nothing prevents you from including more than bsd.port.mk for
local ports.
--
Thomas de Grivel
I must plunge into the water of doubt again and again.
Re: Use of exists() in a Makefile
On 2011-10-04, David Cantrell david.l.cantr...@gmail.com wrote:
My suggestion was to add support to the ports system infrastructure to
allow people an easy way to locally package up stuff from projects that
do not release tarballs.
You could add something like this to your Makefile
create-tarball:
cd ${FULLDISTDIR} \
svn export http://foo ${DISTNAME} \
tar czf ${DISTNAME}${EXTRACT_SUFX} ${DISTNAME}
FETCH_MANUALLY= Use: make create-tarball
Re: Use of exists() in a Makefile
On 10/04/2011 05:00 PM, Christian Weisgerber wrote:
David Cantrelldavid.l.cantr...@gmail.com wrote:
pre-fetch:
.if !exists(${DISTDIR}/${DISTNAME}.tar.gz)
The problem I'm hitting is that !exists() is not working as I
think it should.
.if conditions are evaluated on the spot and...
exists() ignores ${DISTDIR} entirely.
... I bet DISTDIR simply isn't set when the condition is evaluated,
because the variable is only assigned a value later on when bsd.port.mk
is included.
make(1) usually uses lazy evaluation, but .if, .for, and := are
evaluated right when their lines are parsed.
Ah, yeah, that makes sense. I bet that's exactly what's happening.
Thanks,
--
David Cantrell david.l.cantr...@gmail.com
WH6DSN | http://blog.burdell.org/
Re: Use of exists() in a Makefile
On 10/04/2011 05:38 PM, Marc Espie wrote: On Tue, Oct 04, 2011 at 04:43:12PM -0400, David Cantrell wrote: You don't and you don't care about that. I'm talking about extending the infrastructure to support more fetch mechanisms. If people are building from source anyway, especially locally managed ports, if it breaks, they get to keep the pieces. You can't audit things you don't ship, so why care? My suggestion was to add support to the ports system infrastructure to allow people an easy way to locally package up stuff from projects that do not release tarballs. I'm not advocating eliminating checksums on everything, nor am I advocating accepting ports in to the main ports tree that work this way, nor am I advocating destruction of any existing tried and true methods. I'm just pointing out that the infrastructure as it exists could do with a handful of other fetching mechanisms to make life easier for people making local ports. Ports they have no interest in submitting to the main ports tree. Two points: - if stuff such as this is visible from the main tree, some idiots will think they are actually there to be used, and we will have to say no again and again. I would say that's perfect for a port submission review process. The ports infrastructure could certainly contain more functionality than would be allowable in official ports. - stuff such as this won't be used by us, won't work with mirroring tools, and will break sooner or later. Contrarily to what you may think, this is not free. This will require some maintenance. If anything, just to properly ignore the corresponding lines of code. If we keep that feature around, it means some other (actually desireable) feature won't see all that much love. I'm pretty well aware of the costs associated with software development. At any rate, I seem to have hit a thorn. I've viewed the ports infrastructure (the collection of Makefiles and Perl modules) as tools that not only implement the official ports system, but as tools that local administrators can use to package up and maintain very esoteric things. If that's not a goal of the ports infrastructure, it's not a goal and that's fine with me. I will find another way. -- David Cantrell david.l.cantr...@gmail.com WH6DSN | http://blog.burdell.org/
Re: Use of exists() in a Makefile
On 10/06/2011 09:55 AM, Stuart Henderson wrote:
On 2011-10-04, David Cantrelldavid.l.cantr...@gmail.com wrote:
My suggestion was to add support to the ports system infrastructure to
allow people an easy way to locally package up stuff from projects that
do not release tarballs.
You could add something like this to your Makefile
create-tarball:
cd ${FULLDISTDIR} \
svn export http://foo ${DISTNAME} \
tar czf ${DISTNAME}${EXTRACT_SUFX} ${DISTNAME}
FETCH_MANUALLY= Use: make create-tarball
Now that's something that could work in this particular case and it
still fits nicely within the ports infrastructure.
Thanks for the suggestion!
--
David Cantrell david.l.cantr...@gmail.com
WH6DSN | http://blog.burdell.org/
Use of exists() in a Makefile
I'm working on a local port where the source archive is not available
via anything other than svn. I'm trying to use pre-fetch to see if a
checkout of the release I want already exists in /usr/ports/distfiles
and if not, check it out. I'm trying something like this:
V = 1.2.3 # program version
R = 4321 # svn ID
DISTNAME = something-${V}-svn${R}
pre-fetch:
.if !exists(${DISTDIR}/${DISTNAME}.tar.gz)
rm -rf ${WRKDIR}/svn
mkdir -p ${WRKDIR}/svn
( cd ${WRKDIR}/svn ; ${FILESDIR}/checkout.sh ${V} ${R} )
cp ${WRKDIR}/svn/${DISTNAME}.tar.gz ${DISTDIR}
.endif
The block inside the .if/.endif works fine. 'checkout.sh' is something
I wrote to handle checking out the source for the specified version and
svn ID. The problem I'm hitting is that !exists() is not working as I
think it should.
exists() ignores ${DISTDIR} entirely. It works fine if I do:
.if !exists(/usr/ports/distfiles/${DISTNAME}.tar.gz)
If I turn on debugging on make(1), I see that exists() searches the
current directory, /usr/share/mk, and /etc for ${DISTNAME}.tar.gz.
The man page lacks an example usage and nothing I see in
/usr/ports/infrastructure/mk looks to be doing anything different from
what I'm trying to do. There are absolute path checks in there too.
Anyone have any ideas or suggestions?
--
David Cantrell david.l.cantr...@gmail.com
WH6DSN | http://blog.burdell.org/
Re: Use of exists() in a Makefile
On Tue, 4 Oct 2011, David Cantrell wrote:
I'm working on a local port where the source archive is not available via
anything other than svn. I'm trying to use pre-fetch to see if a checkout of
the release I want already exists in /usr/ports/distfiles and if not, check it
out. I'm trying something like this:
Why don't you create a tarball of the checkout and host it?
V = 1.2.3 # program version
R = 4321 # svn ID
DISTNAME = something-${V}-svn${R}
pre-fetch:
.if !exists(${DISTDIR}/${DISTNAME}.tar.gz)
rm -rf ${WRKDIR}/svn
mkdir -p ${WRKDIR}/svn
( cd ${WRKDIR}/svn ; ${FILESDIR}/checkout.sh ${V} ${R} )
cp ${WRKDIR}/svn/${DISTNAME}.tar.gz ${DISTDIR}
.endif
The block inside the .if/.endif works fine. 'checkout.sh' is something I
wrote to handle checking out the source for the specified version and svn ID.
The problem I'm hitting is that !exists() is not working as I think it should.
exists() ignores ${DISTDIR} entirely. It works fine if I do:
.if !exists(/usr/ports/distfiles/${DISTNAME}.tar.gz)
If I turn on debugging on make(1), I see that exists() searches the current
directory, /usr/share/mk, and /etc for ${DISTNAME}.tar.gz.
The man page lacks an example usage and nothing I see in
/usr/ports/infrastructure/mk looks to be doing anything different from what
I'm trying to do. There are absolute path checks in there too.
Anyone have any ideas or suggestions?
--
Antoine
Re: Use of exists() in a Makefile
On 10/04/2011 01:39 PM, Antoine Jacoutot wrote:
On Tue, 4 Oct 2011, David Cantrell wrote:
I'm working on a local port where the source archive is not available via
anything other than svn. I'm trying to use pre-fetch to see if a checkout of
the release I want already exists in /usr/ports/distfiles and if not, check it
out. I'm trying something like this:
Why don't you create a tarball of the checkout and host it?
That's not really the solution I'm after. The project itself does not
have a release engineer and I'm not looking to become one for it. I am
just trying to put together a local port that some other coworkers can
use to build packages of a specific checkout from the svn repo.
V = 1.2.3 # program version
R = 4321 # svn ID
DISTNAME = something-${V}-svn${R}
pre-fetch:
.if !exists(${DISTDIR}/${DISTNAME}.tar.gz)
rm -rf ${WRKDIR}/svn
mkdir -p ${WRKDIR}/svn
( cd ${WRKDIR}/svn ; ${FILESDIR}/checkout.sh ${V} ${R} )
cp ${WRKDIR}/svn/${DISTNAME}.tar.gz ${DISTDIR}
.endif
The block inside the .if/.endif works fine. 'checkout.sh' is something I
wrote to handle checking out the source for the specified version and svn ID.
The problem I'm hitting is that !exists() is not working as I think it should.
exists() ignores ${DISTDIR} entirely. It works fine if I do:
.if !exists(/usr/ports/distfiles/${DISTNAME}.tar.gz)
If I turn on debugging on make(1), I see that exists() searches the current
directory, /usr/share/mk, and /etc for ${DISTNAME}.tar.gz.
The man page lacks an example usage and nothing I see in
/usr/ports/infrastructure/mk looks to be doing anything different from what
I'm trying to do. There are absolute path checks in there too.
Anyone have any ideas or suggestions?
--
David Cantrell david.l.cantr...@gmail.com
WH6DSN | http://blog.burdell.org/
Re: Use of exists() in a Makefile
On Tue, Oct 04, 2011 at 02:11:13PM -0400, David Cantrell wrote: On 10/04/2011 01:39 PM, Antoine Jacoutot wrote: On Tue, 4 Oct 2011, David Cantrell wrote: I'm working on a local port where the source archive is not available via anything other than svn. I'm trying to use pre-fetch to see if a checkout of the release I want already exists in /usr/ports/distfiles and if not, check it out. I'm trying something like this: Why don't you create a tarball of the checkout and host it? That's not really the solution I'm after. The project itself does not have a release engineer and I'm not looking to become one for it. I am just trying to put together a local port that some other coworkers can use to build packages of a specific checkout from the svn repo. Don't use pre-fetch, it's heavily deprecated. In fact, don't override any of pre-fetch, do-fetch, post-fetch. I have code around here somewhere that actually turns these (and other targets that shouldn't *ever* get used into fatal errors. Release engineer or not, we really only support tarballs. make one, get a shell script to make one, or whatever. As far as exists(): it's frowned upon, since it gets evaluated *each time* the Makefile gets used. In most circumstances, it's ways better to use test in the shell fragment associated to a given target.
Re: Use of exists() in a Makefile
On 10/04/2011 02:31 PM, Marc Espie wrote: On Tue, Oct 04, 2011 at 02:11:13PM -0400, David Cantrell wrote: On 10/04/2011 01:39 PM, Antoine Jacoutot wrote: On Tue, 4 Oct 2011, David Cantrell wrote: I'm working on a local port where the source archive is not available via anything other than svn. I'm trying to use pre-fetch to see if a checkout of the release I want already exists in /usr/ports/distfiles and if not, check it out. I'm trying something like this: Why don't you create a tarball of the checkout and host it? That's not really the solution I'm after. The project itself does not have a release engineer and I'm not looking to become one for it. I am just trying to put together a local port that some other coworkers can use to build packages of a specific checkout from the svn repo. Don't use pre-fetch, it's heavily deprecated. In fact, don't override any of pre-fetch, do-fetch, post-fetch. Noted. Please remove information about overriding *-fetch from the bsd.port.mk man page. I have code around here somewhere that actually turns these (and other targets that shouldn't *ever* get used into fatal errors. I do see that in 4.9, the system prevents you from overriding do-fetch. But pre-fetch and post-fetch are still honored. Release engineer or not, we really only support tarballs. make one, get a shell script to make one, or whatever. OK. But this is probably something worth thinking about for future development. I've noticed many upstream projects eliminating tarballs in favor of telling you a git tag to use 'git archive --format=tar' on. While it may not be something anyone cares about for the main ports tree, having the functionality there for people who keep things in /usr/ports/mystuff would probably be useful. I wouldn't mind helping with this either, as it's something I see myself needing more and more. As far as exists(): it's frowned upon, since it gets evaluated *each time* the Makefile gets used. In most circumstances, it's ways better to use test in the shell fragment associated to a given target. Oh well, was trying to use more of make's internal functions than I guess I could reliably count on. Thanks, -- David Cantrell david.l.cantr...@gmail.com WH6DSN | http://blog.burdell.org/
Re: Use of exists() in a Makefile
On Tue, Oct 04, 2011 at 02:44:42PM -0400, David Cantrell wrote:
On 10/04/2011 02:31 PM, Marc Espie wrote:
On Tue, Oct 04, 2011 at 02:11:13PM -0400, David Cantrell wrote:
On 10/04/2011 01:39 PM, Antoine Jacoutot wrote:
On Tue, 4 Oct 2011, David Cantrell wrote:
I'm working on a local port where the source archive is not available via
anything other than svn. I'm trying to use pre-fetch to see if a
checkout of
the release I want already exists in /usr/ports/distfiles and if not,
check it
out. I'm trying something like this:
Why don't you create a tarball of the checkout and host it?
That's not really the solution I'm after. The project itself does
not have a release engineer and I'm not looking to become one for
it. I am just trying to put together a local port that some other
coworkers can use to build packages of a specific checkout from the
svn repo.
Don't use pre-fetch, it's heavily deprecated. In fact, don't override
any of pre-fetch, do-fetch, post-fetch.
Noted. Please remove information about overriding *-fetch from the
bsd.port.mk man page.
Read more closely:
every configuration. Use of {pre,do,post}-fetch hooks is
strongly discouraged, and will probably be removed in the
near future, as this makes mirroring of distfiles very
complicated. See CHECKSUMFILES, CDROM_SITE, DISTDIR,
OK. But this is probably something worth thinking about for future
development. I've noticed many upstream projects eliminating
tarballs in favor of telling you a git tag to use 'git archive
--format=tar' on. While it may not be something anyone cares about
for the main ports tree, having the functionality there for people
who keep things in /usr/ports/mystuff would probably be useful.
Nope. Not good. checksums. How do you prevent people tampering from
upstream and introduce trojan horses ?
It's not like this never happened. We caught at least 2 such issues
thanks to the checksums in distinfo.
Re: Use of exists() in a Makefile
On 10/04/2011 04:27 PM, Marc Espie wrote:
On Tue, Oct 04, 2011 at 02:44:42PM -0400, David Cantrell wrote:
On 10/04/2011 02:31 PM, Marc Espie wrote:
On Tue, Oct 04, 2011 at 02:11:13PM -0400, David Cantrell wrote:
On 10/04/2011 01:39 PM, Antoine Jacoutot wrote:
On Tue, 4 Oct 2011, David Cantrell wrote:
I'm working on a local port where the source archive is not available via
anything other than svn. I'm trying to use pre-fetch to see if a checkout of
the release I want already exists in /usr/ports/distfiles and if not, check it
out. I'm trying something like this:
Why don't you create a tarball of the checkout and host it?
That's not really the solution I'm after. The project itself does
not have a release engineer and I'm not looking to become one for
it. I am just trying to put together a local port that some other
coworkers can use to build packages of a specific checkout from the
svn repo.
Don't use pre-fetch, it's heavily deprecated. In fact, don't override
any of pre-fetch, do-fetch, post-fetch.
Noted. Please remove information about overriding *-fetch from the
bsd.port.mk man page.
Read more closely:
every configuration. Use of {pre,do,post}-fetch hooks is
strongly discouraged, and will probably be removed in the
near future, as this makes mirroring of distfiles very
complicated. See CHECKSUMFILES, CDROM_SITE, DISTDIR,
Yes, I saw that. It does say probably though. And it only says
discouraged, not deprecated.
Many would interpret the text to mean that while {pre,do,post}-fetch
hooks are still there, it's probably there for backwards compatibility
and you probably want to look at another way to do what you're trying to
do. But it doesn't indicate that the functionality will definitely be
removed or that it should not be used at all.
OK. But this is probably something worth thinking about for future
development. I've noticed many upstream projects eliminating
tarballs in favor of telling you a git tag to use 'git archive
--format=tar' on. While it may not be something anyone cares about
for the main ports tree, having the functionality there for people
who keep things in /usr/ports/mystuff would probably be useful.
Nope. Not good. checksums. How do you prevent people tampering from
upstream and introduce trojan horses ?
You don't and you don't care about that. I'm talking about extending
the infrastructure to support more fetch mechanisms. If people are
building from source anyway, especially locally managed ports, if it
breaks, they get to keep the pieces. You can't audit things you don't
ship, so why care?
My suggestion was to add support to the ports system infrastructure to
allow people an easy way to locally package up stuff from projects that
do not release tarballs. I'm not advocating eliminating checksums on
everything, nor am I advocating accepting ports in to the main ports
tree that work this way, nor am I advocating destruction of any existing
tried and true methods. I'm just pointing out that the infrastructure
as it exists could do with a handful of other fetching mechanisms to
make life easier for people making local ports. Ports they have no
interest in submitting to the main ports tree.
It's not like this never happened. We caught at least 2 such issues
thanks to the checksums in distinfo.
Right, and I'm not talking about changing anything in the main ports
tree to drop checksums. I'm merely suggesting introducing additional
mechanisms by which people can easily package things on their own
systems from projects that do not release tarballs. I'm not advocating
AGAINST using checksums.
--
David Cantrell david.l.cantr...@gmail.com
WH6DSN | http://blog.burdell.org/
Re: Use of exists() in a Makefile
David Cantrell david.l.cantr...@gmail.com wrote:
pre-fetch:
.if !exists(${DISTDIR}/${DISTNAME}.tar.gz)
The problem I'm hitting is that !exists() is not working as I
think it should.
.if conditions are evaluated on the spot and...
exists() ignores ${DISTDIR} entirely.
... I bet DISTDIR simply isn't set when the condition is evaluated,
because the variable is only assigned a value later on when bsd.port.mk
is included.
make(1) usually uses lazy evaluation, but .if, .for, and := are
evaluated right when their lines are parsed.
--
Christian naddy Weisgerber na...@mips.inka.de
Re: Use of exists() in a Makefile
On Tue, Oct 04, 2011 at 04:43:12PM -0400, David Cantrell wrote: You don't and you don't care about that. I'm talking about extending the infrastructure to support more fetch mechanisms. If people are building from source anyway, especially locally managed ports, if it breaks, they get to keep the pieces. You can't audit things you don't ship, so why care? My suggestion was to add support to the ports system infrastructure to allow people an easy way to locally package up stuff from projects that do not release tarballs. I'm not advocating eliminating checksums on everything, nor am I advocating accepting ports in to the main ports tree that work this way, nor am I advocating destruction of any existing tried and true methods. I'm just pointing out that the infrastructure as it exists could do with a handful of other fetching mechanisms to make life easier for people making local ports. Ports they have no interest in submitting to the main ports tree. Two points: - if stuff such as this is visible from the main tree, some idiots will think they are actually there to be used, and we will have to say no again and again. - stuff such as this won't be used by us, won't work with mirroring tools, and will break sooner or later. Contrarily to what you may think, this is not free. This will require some maintenance. If anything, just to properly ignore the corresponding lines of code. If we keep that feature around, it means some other (actually desireable) feature won't see all that much love.
