Re: Test::Smoke failing to test most recent commit to Perl 5 blead
On 01/14/2017 10:10 AM, James E Keenan wrote:
On 01/13/2017 11:25 PM, James E Keenan wrote:
The most recent smoke testing of Perl 5 blead I have conducted in my
FreeBSD-10.3 VM is that found at:
Here is an additional instance of the problem, which I have reported as:
https://rt.cpan.org/Ticket/Display.html?id=120008
#
This is a bug report which provides an additional instance of a problem
originally reported to the perl.qa mailing list/newsgroup (thread
starting at: http://www.nntp.perl.org/group/perl.qa/2017/01/msg13734.html).
I have a VM of FreeBSD-10.3 in which I customarily test blead and/or
smoke-me branches in the Perl 5 core distribution. When I wish to test
a smoke-me branch, I set the first line of
'p5smoke/install/smokecurrent.gitbranch' to the name of that branch as
it appears on remote 'origin'. I then call 'cd ~/p5smoke/install; sh
./smokecurrent.sh' and let Test-Smoke do its thing.
On Jan 26 at about 2200 localtime (Jan 27 0300 UTC) I conducted a smoke
test run on the 'smoke-me/jkeenan/130635-storable' branch:
#
$ grep -n smoke_branch jsnd6115793d6cc41755a3ed4baaa38d30653656f41.jsn
85: "smoke_branch" : "smoke-me/jkeenan/130635-storable",
#
The smoke test run -- whose report you can see at
http://perl5.test-smoke.org/report/53530 -- resulted in FAILs:
#
Failures: (common-args) -Doptimize="-O2 -pipe -fstack-protector
-fno-strict-aliasing"
[stdio/perlio] -DDEBUGGING -Duseithreads
../t/porting/exec-bit.t.FAILED
97-98
../t/porting/podcheck.t.FAILED
183
#
These failures had nothing to do with the branch being smoked. That
branch had been created minutes earlier by pushing one commit on top of
Perl 5 blead. Those failures occurred in blead, originating in this commit:
#
commit 1f74a12bfce8b82144fa7fef29618af1e8023298
Author: Christian Millour
Date: Thu Jan 19 11:49:03 2017 +0100
document nature and use of $a and $b in sort()
Signed-off-by: Abigail
#
I switched to a separate git checkout directory that I maintain on the
same VM, updated and tested blead, confirmed and corrected the two test
failures and pushed the corrections to blead.
#
commit 475b224feea308464e18cc6bef788e7a152afa51
Author: James E Keenan
AuthorDate: Thu Jan 26 22:44:03 2017
Commit: James E Keenan
CommitDate: Thu Jan 26 22:45:26 2017
Correct file mode and line lengths to keep porting tests happy.
#
This morning I returned to that VM, moved into p5smoke/install and
updated smokecurrent.gitbranch to have its first line read 'blead':
#
$ head -5 smokecurrent.gitbranch
blead
smoke-me/jkeenan/130635-storable
smoke-me/jkeenan/revert-cpan-118470
smoke-me/jkeenan/77934-threads
smoke-me/jkeenan/petdance/130519-remove-two-unused-macros
#
I then called 'sh ./smokecurrent.sh' to kick off another smoke test run,
this time for 'blead'. At that point additional commits had been made
to blead. HEAD was:
#
commit 9a7b7fba267b8910dbea2920dd675030f796320c
Author: Yves Orton
AuthorDate: Fri Jan 27 04:23:05 2017
Commit: Yves Orton
CommitDate: Fri Jan 27 04:23:24 2017
#
When the smoke test run completed, however, I was surprised to see that
the commit being tested -- HEAD in the p5smoke/perl-current and
p5smoke/git-perl directories -- was still commit
1f74a12bfce8b82144fa7fef29618af1e8023298 -- the commit which caused the
porting test errors and for which I had already pushed corrections to blead!
Here is the 'sysinfo' element from
jsn1f74a12bfce8b82144fa7fef29618af1e8023298.jsn:
#
"sysinfo" : {
"lang" : null,
"smoke_date" : "2017-01-27 08:11:48 -0500",
"git_id" : "1f74a12bfce8b82144fa7fef29618af1e8023298",
"git_describe" : "v5.25.9-49-g1f74a12",
"osname" : "freebsd",
"smoker_version" : "0.046",
"test_jobs" : "8",
"smoke_revision" : "1.70",
"osversion" : "10.3-RELEASE",
"username" : "jkeenan",
"lc_all" : null,
"smoke_branch" : "blead",
"smoke_perl" : "5.20.3",
"reporter" : null,
"cpu_description" : "Intel(R) Core(TM) i5-4200M CPU @ 2.50GHz",
"hostname" : "localhost",
"cpu_count" : "1",
"user_note" : "Occasional failures observed in long-running tests
in:\nt/re/speed.t\nt/re/speed_thr.t\n
cpan/Memoize/t/expmod_t.t\ndist/Time-HiRes/t/alarm.t\n",
"smoke_version" : "1.70",
"reporter_version" : "0.053",
"duration" : 2349,
"perl_id" : "5.25.10",
"config_count" : 1,
"architecture" : "amd64"
},
#
Not surprisingly, since 'blead' had not been moved forward from the
commit at which the porting test errors were introduced, those errors
were still present:
#
"failures" : [
{
"extra" : [
"97-98"
],
"test" : "../t/porting/
Re: Test::Smoke failing to test most recent commit to Perl 5 blead
On 28 Jan 2017 6:32 a.m., "James E Keenan" wrote: > On 01/14/2017 10:10 AM, James E Keenan wrote: > Here is the first part of the smokecurrent.log for this run -- with some > comments: > > # > [2017-01-27 08:11:22-0500] Read configuration from: > /usr/home/jkeenan/p5smoke/install/smokecurrent_config > [2017-01-27 08:11:22-0500] Commitlevel before sync: > d6115793d6cc41755a3ed4baaa38d30653656f41 > > # d611579 was HEAD in the previous branch being smoked: > smoke-me/jkeenan/130635-storable > > [2017-01-27 08:11:22-0500] ==> Starting synctree > [2017-01-27 08:11:22-0500] Reading branch to smoke from: > '/usr/home/jkeenan/p5smoke/install/smokecurrent.gitbranch' > [2017-01-27 08:11:22-0500] In pwd(/usr/home/jkeenan/p5smoke/git-perl) > running: > [2017-01-27 08:11:22-0500] qx[/usr/local/bin/git pull --all] > From git://perl5.git.perl.org/perl >1f74a12..9a7b7fb blead -> origin/blead > [2017-01-27 08:11:37-0500] Fetching origin > [2017-01-27 08:11:37-0500] Already up-to-date. > [2017-01-27 08:11:37-0500] In pwd(/usr/home/jkeenan/p5smoke/git-perl) > running: > [2017-01-27 08:11:37-0500] qx[/usr/local/bin/git remote prune origin] > [2017-01-27 08:11:38-0500] In pwd(/usr/home/jkeenan/p5smoke/git-perl) > running: > [2017-01-27 08:11:38-0500] qx[/usr/local/bin/git checkout blead > [2017-01-27 08:11:38-0500] 2>&1] > Switched to branch 'blead' > [2017-01-27 08:11:41-0500] Your branch is behind 'origin/blead' by 7 > commits, and can be fast-forwarded. > [2017-01-27 08:11:41-0500] (use "git pull" to update your local branch) > > # Note: No indication that 'git pull' was actually run! Why not? > > [2017-01-27 08:11:41-0500] In pwd(/usr/home/jkeenan/p5smoke/perl-current) > running: > [2017-01-27 08:11:41-0500] qx[/usr/local/bin/git reset --hard] > > # Note: Although 'man git-reset' is not explicit about this, we can > probably assume > # that 'git reset --hard' with no resets to HEAD -- i.e., no > update to checkout. > > Yes. It throws away any changes to the current branch. That should say git reset --hard origin/blead > [2017-01-27 08:11:43-0500] HEAD is now at d611579 Fix stack buffer > overflow in deserialization of hooks. > # > > So why does Test-Smoke not update the branch being tested in cases like > this? > > This is an educated guess: whoever wrote that code did not understand git well and got confused about what git pull does, and what git pull --all do. Seeing git pull in a script like this is a red flag, seeing --all is a bigger red flag for me. ( It does not "pull all", it does a fetch against all remotes.) IMO git pull is not really suitable for scripting as it can trigger a rebase or merge, and trigger an editor. I would expect to see a sequence like this: git remote update -p git checkout $branch git reset --hard origin/$branch or like this: git remote prune --all git fetch --all git checkout $branch git reset --hard origin/$branch or even like this: git fetch origin git checkout $branch git reset --hard origin/$branch The difference between the variants being about whether the code should be managing multiple upstream repos or not. The --all introduces ambiguity about this, as it means "fetch code from --all remotes", so one might guess someone wanted to support multiple upstreams. On the other hand, my guess is that the person who coded it to do a 'git pull --all' thought that the -all would update --all branches, which it does not. Bolstering this view is that it seems to make little sense to smoke branches from multiple upstreams, especially for Perl. I would have expected only the canonical branches in the master upstream repo should be smoked, so the --all probably was a bug. What appears to be happening is that when the pull is executed it is in a different branch, so it fetches and then updates *THAT* branch only (via a merge!). When it then checks-out the new branch branch it is on an old commit, and requires a reset to the latest code. I would recommend that the smoke scripts do not use pull *ever*. They should never ever modify code, so doing a pull makes no sense. git pull is basically a smart wrapper around git fetch + git merge. A smoker should *NEVER* be doing git-merge, so it should *never* be using git-pull. Note, I am fully aware that under perfect circumstances you /can/ script this using git pull. I would recommend not to. First it is confusing. Second, if circumstances are less than perfect then it could lead to unwanted behavior. For instance someone tinkers in a branch used by the smoker, then one could imaging the script breaking because of uncommitted changes, or breaking because pull has triggered a merge which requires a edited text message. FWIW, i have in the past recommended that new users to git do NOT use "git pull", until they have mastered using "git fetch" and "git rebase" or "git merge" as independent commands. Only once having experience of the three basic commands should people use git pull. Since it is a wrapper around three dis
