Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[Eli Schwartz]

On 08/07/2017 10:12 PM, David C. Rankin wrote:
> On 08/07/2017 09:36 AM, Eli Schwartz wrote:
>> You could also use `makepkg --noextract` which assumes you applied the
>> fix by hand rather than in the PGBUILD, which is probably safer as it
>> skips prepare(), and in theory re-running configure should not cause
>> Make to expire all object files (though with autotools who really knows...)
>>
> 
> That is one thing I didn't consider. It has always been beaten into my head
> that you don't touch the original source and you take the time to create a
> .patch to be applied though prepare().
> 
> I guess this is one instance where it would make sense to actually edit the
> files in the source-tree directly in order to skip prepare.
> 
> Thanks.

You should do both. Set it in prepare() to save yourself the effort in
the future, and for your local run you can do it by hand to save time
when not doing a clean rebuild.

prepare() is not idempotent, its value lies in making sure you can
replicate what you did next week when running `makepkg -C`.

-- 
Eli Schwartz



signature.asc
Description: OpenPGP digital signature

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[Eli Schwartz]

On 08/07/2017 10:06 PM, David C. Rankin wrote:
> On 08/07/2017 06:06 AM, Christoph Gysin via arch-general wrote:
>> Not the answer to your question, but you might consider configuring
>> /etc/makepkg.conf to use ccache to speed up rebuilds. Also set
>> MAKEFLAGS=-j4 or similar to make sure you are using all available
>> cores.
> 
> Thanks Cristoph,
> 
>   How do you know whether a package is capable of being built in parallel?
> I've run into problems before with big builds when building in parallel. Is it
> just a 'try and see if it works' issue, or is there some general rule-of-thumb
> you can use?

Generally, you try and see if it works. If it doesn't, the build rules
don't properly specify dependencies, and you file an upstream bug
report, which may or may not get fixed, and you make sure the PKGBUILD
maintainer specifically resets MAKEFLAGS back to -j1 (this is the only
time it is okay to specify the number of make jobs to run in parallel,
if you are resetting back to the defaults because a package doesn't
compile with custom MAKEFLAGS in makepkg.conf).

-- 
Eli Schwartz



signature.asc
Description: OpenPGP digital signature

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[David C. Rankin]

On 08/07/2017 09:36 AM, Eli Schwartz wrote:
> You could also use `makepkg --noextract` which assumes you applied the
> fix by hand rather than in the PGBUILD, which is probably safer as it
> skips prepare(), and in theory re-running configure should not cause
> Make to expire all object files (though with autotools who really knows...)
> 

That is one thing I didn't consider. It has always been beaten into my head
that you don't touch the original source and you take the time to create a
.patch to be applied though prepare().

I guess this is one instance where it would make sense to actually edit the
files in the source-tree directly in order to skip prepare.

Thanks.

> Aside: repackage does not do what you want, since it doesn't rebuild
> anything. Granted, Makefiles usually have the install target depend on
> the default target, so it would probably end up being built too... in
> fakeroot...

Yes, the reference to --repackage was just a way of describing the state of
the build directory. Repackaging the same executables hoping something changed
is as productive as trying to destroy a shot-put with a rubber hammer...


-- 
David C. Rankin, J.D.,P.E.



signature.asc
Description: OpenPGP digital signature

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[David C. Rankin]

On 08/07/2017 06:06 AM, Christoph Gysin via arch-general wrote:
> Not the answer to your question, but you might consider configuring
> /etc/makepkg.conf to use ccache to speed up rebuilds. Also set
> MAKEFLAGS=-j4 or similar to make sure you are using all available
> cores.

Thanks Cristoph,

  How do you know whether a package is capable of being built in parallel?
I've run into problems before with big builds when building in parallel. Is it
just a 'try and see if it works' issue, or is there some general rule-of-thumb
you can use?

-- 
David C. Rankin, J.D.,P.E.

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[ProgAndy]

Am 07.08.2017 um 09:39 schrieb David C. Rankin:


If it wasn't clear, I have already built the gtk2 package yesterday to
--enable-debug=yes so I have all of the files in a state I could call
--repackge on, except for the gtk/gtkrecentchooser.c file with the one line
change. I was wondering if there was a way to avoid the full 15 minute rebuild
of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then
--repackage?

You can use makepkg with the -e (--noextract) option to rebuild from the 
existing source tree. If the configure step forces a full rebuild, you 
can comment it out in the PKGBUILD after the initial build.



--
Andy

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[Eli Schwartz]

On 08/07/2017 03:39 AM, David C. Rankin wrote:
> On 08/06/2017 10:23 PM, David C. Rankin wrote:
>> All,
>>
>>   There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c
>> file. (it was a one-line fix where a variable needed reset to 0). Is there 
>> any
>> way I can configure makepkg to behave as if the gcc -MP -MD options had been
>> given to have it only recompile the one patched source without rebuilding the
>> entire package -- again?
>>
>>   I've been through the wiki and the man page, and other than --repackage,
>> there doesn't seem to be anything that would work (or I'm just missing it).
> 
> If it wasn't clear, I have already built the gtk2 package yesterday to
> --enable-debug=yes so I have all of the files in a state I could call
> --repackge on, except for the gtk/gtkrecentchooser.c file with the one line
> change. I was wondering if there was a way to avoid the full 15 minute rebuild
> of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then
> --repackage?

I think you're misunderstanding what -MP does.

As for rebuilding only the one source, makepkg source extraction should
result in extracting files with the same initial timestamp as the last
time they were extracted. Although if you are using the gtk2 PKGBUILD
from the ABS, it uses git anyway... So Make should consider all objects
to be up-to-date, except the one you have sed'ed.

You could also use `makepkg --noextract` which assumes you applied the
fix by hand rather than in the PGBUILD, which is probably safer as it
skips prepare(), and in theory re-running configure should not cause
Make to expire all object files (though with autotools who really knows...)

Aside: repackage does not do what you want, since it doesn't rebuild
anything. Granted, Makefiles usually have the install target depend on
the default target, so it would probably end up being built too... in
fakeroot...

-- 
Eli Schwartz



signature.asc
Description: OpenPGP digital signature

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[respiranto]

On 2017-08-07 09:39, David C. Rankin wrote:
> On 08/06/2017 10:23 PM, David C. Rankin wrote:
>> All,
>>
>>   There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c
>> file. (it was a one-line fix where a variable needed reset to 0). Is there 
>> any
>> way I can configure makepkg to behave as if the gcc -MP -MD options had been
>> given to have it only recompile the one patched source without rebuilding the
>> entire package -- again?
>>
>>   I've been through the wiki and the man page, and other than --repackage,
>> there doesn't seem to be anything that would work (or I'm just missing it).
> 
> If it wasn't clear, I have already built the gtk2 package yesterday to
> --enable-debug=yes so I have all of the files in a state I could call
> --repackge on, except for the gtk/gtkrecentchooser.c file with the one line
> change. I was wondering if there was a way to avoid the full 15 minute rebuild
> of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then
> --repackage?
> 

That should be precisely the job of make. All the other preparation
steps such as ./configure or patching should not harm either, likely
rather be necessary.

Hence, you could just try to rerun makepkg, providing that the $srcdir
is still in the state after the last successful build.

In fact, I already had problems doing this. Suppose a sed script
replacing 'x' by 'xx', inline. In such cases one might consider using
the --noprepare flag.

In any case, it is a good idea to read the PKGBUILD (as always) and
possibly modify it or even do all the work manually.

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[Christoph Gysin via arch-general]

On Mon, Aug 7, 2017 at 10:39 AM, David C. Rankin
 wrote:
> If it wasn't clear, I have already built the gtk2 package yesterday to
> --enable-debug=yes so I have all of the files in a state I could call
> --repackge on, except for the gtk/gtkrecentchooser.c file with the one line
> change. I was wondering if there was a way to avoid the full 15 minute rebuild
> of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then
> --repackage?

Not the answer to your question, but you might consider configuring
/etc/makepkg.conf to use ccache to speed up rebuilds. Also set
MAKEFLAGS=-j4 or similar to make sure you are using all available
cores.

-- 
echo mailto: NOSPAM !#$.'<*>'|sed 's. ..'|tr "<*> !#:2" org@fr33z3

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[LoneVVolf]

On 07-08-17 09:39, David C. Rankin wrote:
I was wondering if there was a way to avoid the full 15 minute rebuild

of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then
--repackage?



I don't know a way to do that, but do you have ccache installed ?

On my system it's set to use max 20G on a HDD .
When building things like mesa or llvm repeatedly with small changes 
this can shaves minutes of the build time.


LW

https://wiki.archlinux.org/index.php/Ccache

Re: [arch-general] makepkg - any way to recompile only newly patched files in large packages?

[David C. Rankin]

On 08/06/2017 10:23 PM, David C. Rankin wrote:
> All,
> 
>   There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c
> file. (it was a one-line fix where a variable needed reset to 0). Is there any
> way I can configure makepkg to behave as if the gcc -MP -MD options had been
> given to have it only recompile the one patched source without rebuilding the
> entire package -- again?
> 
>   I've been through the wiki and the man page, and other than --repackage,
> there doesn't seem to be anything that would work (or I'm just missing it).

If it wasn't clear, I have already built the gtk2 package yesterday to
--enable-debug=yes so I have all of the files in a state I could call
--repackge on, except for the gtk/gtkrecentchooser.c file with the one line
change. I was wondering if there was a way to avoid the full 15 minute rebuild
of all of gtk2 and just compile gtkrecentchooserdefault.c to object and then
--repackage?

-- 
David C. Rankin, J.D.,P.E.

[arch-general] makepkg - any way to recompile only newly patched files in large packages?

[David C. Rankin]

All,

  There was a fix for a gtk2 bug I filed regarding the recentchooserdefault.c
file. (it was a one-line fix where a variable needed reset to 0). Is there any
way I can configure makepkg to behave as if the gcc -MP -MD options had been
given to have it only recompile the one patched source without rebuilding the
entire package -- again?

  I've been through the wiki and the man page, and other than --repackage,
there doesn't seem to be anything that would work (or I'm just missing it).

-- 
David C. Rankin, J.D.,P.E.