On Fri, Jan 7, 2011 at 12:40, Henning Schild
<henn...@plan9.bell-labs.com> wrote:
> On Fri, 7 Jan 2011 10:51:56 +0100
> Ciprian Dorin Craciun <ciprian.crac...@gmail.com> wrote:
>
>> :) I've kind of feared that this is the reason... :)
>>
>> But still how do people handle the issue?
>
>
> I guess in most cases it is ok to ignore the slight waste of CPU-time.
> And i guess people just ignore it. After all it costs less than a
> second for each of these targets. If your project it full of them and
> they are deps for bigger targets you may want to add "sleep x.y" to the
> rules. That way it will maybe take more time but will idle instead of
> wasting CPU-time. When using the sleeps adjusting NPROC might help to do
> something useful while sleeping.
>
> Another way i could think of is adjusting the mtimes using touch. So
> touch the sources -1s or the target +1s after creation of the target.
> Depending on the graph that might cause other trouble i guess.
>
> Henning
Thank you all for the feedback!
Indeed having some very quick targets rebuilt again isn't that a
big of an issue, nor is waiting 1 second at the end of each recipe.
(The solution with touching the files out of recipes is not applicable
as it interferes with the dependency graph...)
But the above statement is true unless you have about 367 targets
(for quite a small project -- only 2 tiny and 1 larger Erlang
applications), which when built takes about 45 seconds (with
NPROC=16), and the second time (without touching a single file in the
source directory) takes about exactly the same amount (still 45
seconds), the third time goes down to 44 seconds, then 43 seconds,
then 7 seconds, then 1 second, then 0.6 seconds, then 0.2 seconds,
then again 0.2 seconds, again 0.2 seconds, again 0.2, again 0.2, again
0.2, and finally after ONLY 13 builds it finds out that nothing is to
be made...
So unfortunately neither of the solutions (both waiting, or
touching) are quite reasonable for me...
As a consequence I've "hacked" `mk` to take into consideration
sub-second timestamps... (And believe me the term "hacked" is most
appropriated as my solution isn't quite that clean.) :) (For those
interested I've put the patch at the end. It seems that I needed only
to hack the `mkmtime` function in `unix.c` to do the following: I
assume that the oldest file that I care to take into account is not
older than 24 hours from now, any file older than that is capped at 24
hours ago; I also assume that any file newer than one hour from now
(any over is capped again); thus I use the signed long not as seconds
from the epoch but as milliseconds starting from 24 hours ago... The
patch works only on Linux, by using the non portable (??)
(undocumented) `st_mtim` (and not `st_mtime`) `stat` field... Any one
knows a portable Unix way of doing this?)
BTW... People might wonder how come I have 367 targets (with 1221
prerequisites) for such a small project? :) The answers is I don't
write the `mk` script by hand, but I've written a small Scheme
application that just generates the `mk` script based on descriptions
like the following. (Thus the resulting `mk` script is quite
exhaustive with quite tight dependencies and doesn't use
meta-rules...) :)
So just out of curiosity are there any `mk` script generators out there?
Ciprian.
~~~~
(vbs:require-erlang)
(vbs:define-erlang-application 'rabbit
erl: "\\./(rabbitmq-server--latest/src|generated)/.*\\.erl"
hrl: "\\./(rabbitmq-server--latest/include|generated)/.*\\.hrl"
additional-ebin: "\\./generated/rabbit\\.app")
(vbs:define-erlang-application 'rabbit_common
erl:
"\\./(rabbitmq-server--latest/src|generated)/(rabbit_writer|rabbit_reader|rabbit_framing_amqp_0_8|rabbit_framing_amqp_0_9_1|rabbit_framing_channel|rabbit_basic|rabbit_binary_generator|rabbit_binary_parser|rabbit_channel|rabbit_exchange_type|rabbit_misc|rabbit_net|rabbit_heartbeat|rabbit_msg_store_index|gen_server2|priority_queue|supervisor2)\\.erl"
hrl: "\\./(rabbitmq-server--latest/include|generated)/.*\\.hrl"
additional-ebin: "\\./generated/rabbit_common\\.app")
(vbs:define-erlang-application 'amqp_client
dependencies: 'rabbit_common
erl: "\\./rabbitmq-erlang-client--latest/src/.*\\.erl"
hrl: "\\./rabbitmq-erlang-client--latest/include/.*\\.hrl"
additional-ebin: "\\./generated/amqp_client\\.app")
~~~~
~~~~
diff -u ./mk-orig/unix.c ./mk-timeres/unix.c
--- ./mk-orig/unix.c 2011-01-02 22:52:50.000000000 +0200
+++ ./mk-timeres/unix.c 2011-01-09 16:43:32.150837213 +0200
@@ -329,13 +329,33 @@
}
}
+#define TIMERES_SEC ((unsigned long) 1000)
+#define TIMERES_NANO ((unsigned long) 1000 * 1000)
+#define TIMEBASE ((unsigned long) 1024)
+#define TIMESPAN_SEC ((((((unsigned long) 1) << 31) - 1 - TIMEBASE -
1) / TIMERES_SEC / 3600) * 3600)
+
unsigned long
mkmtime(char *name)
{
struct stat st;
+ static unsigned long long ref_time = 0;
+ unsigned long long abs_time;
+ unsigned long rel_time;
if(stat(name, &st) < 0)
return 0;
- return st.st_mtime;
+ abs_time = ((unsigned long long) (st.st_mtim.tv_sec)) * TIMERES_SEC
+ (st.st_mtim.tv_nsec / TIMERES_NANO);
+
+ if (ref_time == 0)
+ ref_time = ((unsigned long long) (time((void*)0) -
TIMESPAN_SEC)) *
TIMERES_SEC;
+
+ if (abs_time <= ref_time)
+ rel_time = TIMEBASE;
+ else if (abs_time >= (ref_time + (TIMESPAN_SEC * TIMERES_SEC)))
+ rel_time = TIMEBASE + (TIMESPAN_SEC * TIMERES_SEC);
+ else
+ rel_time = TIMEBASE + (abs_time - ref_time);
+
+ return rel_time;
}
~~~~
