On 11/08/2017 02:52 AM, Merijn van den Kroonenberg wrote:
On 11/07/2017 01:07 PM, Merijn van den Kroonenberg wrote:
On 11/07/2017 10:24 AM, Merijn van den Kroonenberg wrote:
Merijn,
I patched the generate-new-scores.sh locally on sa-vm1 using your
patch
file with a slight adjustment. I changed the copied file name to
"72_active_before_grep.cf" just to make it a little more obvious. We
will see how it looks tomorrow in the tmp working area on sa-vm1 and
I
will reply with the results.
I looked at todays tmp/generate-new-scores/trunk-new-rules-set0 and I
do
not see this patch applied. Can you even do uncommitted changes to
code
of
masses, or will it just be removed as a fresh checkout is done each
time?
It was setup but I too noticed that it didn't create the
72_active_before_grep.cf but accidentally forgot about following up on
that. I will check on that later this evening to get this in place
before the next run in about 10 hours.
Ok thanks, its no longer high priority, I think I can do without for
debugging now. But in the longer run we can use it to decide if we can
get
rid of the grep statement altogether (if 72_active_before_grep.cf always
equals 72_active.cf then it has no use).
I just brought it up because I wondered where that code went, but guess
you removed it when it didn't work.
Well... My local patch is still in place. This is lines 195-206 of the
current trunk/masses/rule-update-score-gen/generate-new-scores.sh:
What is the full path? The question is in which working copy did you apply
this change (what is it used for). Because in the working directories used
for score generation it isn't. Thats in
/usr/local/spamassassin/automc/tmp/generate-new-scores/trunk-new-rules-set0
(each set having its fully checked out working copy)
In generate-new-scores.sh the workingcopies are done each night completely
from scratch (old one removed, new one checked out). So no uncommitted
code can be used in the score generation part.
/usr/local/spamassassin/automc/svn/trunk/build/mkupdates/do-stable-update-with-scores
should be calling
/usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/do-nightly-rescore-example.sh
which finally calls the locally modified and not committed script that
is intended to create our 72_active_before_grep.cf but it's not some
something is definitely not as I expected
/usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/generate-new-scores.sh
All of those script should be checking out fresh copies of everything
they work with into
/usr/local/spamassassin/automc/tmp
=== Line 195 ===
date
echo "[ generating active ruleset via make ]"
perl Makefile.PL < /dev/null || exit $?
make > make.out 2>&1 || exit $?
# temp debug copy (investigate truncation issue)
cp rules/72_active.cf 72_active_before_grep.cf
# strip scores from new rules so that the garescorer can set them
grep -v ^score rules/72_active.cf > rules/72_active.cf-scoreless
mv -f rules/72_active.cf-scoreless rules/72_active.cf
=== Line 206 ===
So the 72_active_before_grep.cf should exist in the tmp area the past
several nights. Hmmm.
I updated the do-stable-update-with-score cron entry to output to a log
file here in a few hours to see if we can get some helpful logging.
Okay now I see, all code up to the generate-new-scores.sh script itself is
ran from:
/usr/local/spamassassin/automc/svn/
but everything called inside generate-new-scores.sh runs from
/usr/local/spamassassin/automc/tmp/generate-new-scores/trunk-new-rules-set0
and
/usr/local/spamassassin/automc/tmp/generate-new-scores/trunk-new-rules-set1
So I guess /usr/local/spamassassin/automc/svn/ is updated to latest, but
with the svn update command local changes are not overwritten so
everything in there you can test without committing.
Thank what I think too.
Everything run from generate-new-scores.sh however is freshly checked out
so it can only run code which has been committed.
That's how it's worked in the past but I could be wrong. My memory
isn't what it used to be. :)
I guess this might have frustrated some of your debugging in the past :(
Dave
