PLEASE FIX THESE APPARENTLY TRIVIAL PROBLEMS.
I ran the tests early this morning. I compared the results with the ones
from my run a week ago. Some improvements, some new failures.
First comes my summary analysis of the new problems. (Heaven knows we
need the same analysis of the old problems.) Almost all of these
should be fixed right away since they are trivial and generate chaff.
After that, I include the diff of the testing/utils/kvmresults.py
outputs, interspersed with snippits of log file diffs.
(1)
testing/pluto/ikev2-x509-23-no-ca failed east:output-different
testing/pluto/nss-cert-nosecret failed east:output-different
west:output-different
testing/pluto/nss-cert-ocsp-02 failed east:output-different
testing/pluto/nss-cert-ocsp-02-ikev2 failed east:output-different
testing/pluto/nss-cert-ocsp-07-nourl failed east:output-different
testing/pluto/nss-cert-ocsp-08-post failed east:output-different
testing/pluto/nss-cert-badca-01 failed east:output-different
Somehow some cert errors are duplicate.
(2) [endemic: not likely to be easy to fix]
testing/pluto/xauth-pluto-14 failed road:output-different
testing/pluto/xauth-pluto-26 failed road:output-different
As always, some data (ping?) packets get dropped, apparently
non-deterministically.
(3)
testing/pluto/newoe-21-liveness-clear failed east:output-different
Timing changed withing log message. Is this just non-determinism?
Does this need sanitization?
(4)
testing/pluto/newoe-27-replace-sa-auth-authnull failed road:output-different
EVENT_SA_REKEY is now used where EVENT_SA_REPLACE was previously used.
Reference log should be updated.
(6)
testing/pluto/interop-ikev1-strongswan-01-xauth failed east:output-different
west:output-different
+strongswan 5.7.1 must be installed
What's this about? I seem to have 5.7.2 installed.
- should that not be good enough?
- should the message not report what version is actually installed?
Test requires strongswan 5.7.1 but 5.7.2 is installed
(7)
testing/pluto/nss-cert-nosecret failed east:output-different
west:output-different
Looks like a sanitizer change without an update to reference logs.
(8)
testing/pluto/nss-cert-ocsp-03-strict failed west:output-different
Mystery change, probably involving message Ids.
================================================================
< testing/pluto/enumcheck-01 failed west:output-different
> testing/pluto/enumcheck-01 passed
< testing/pluto/ikev2-algo-06-aes-aes_xcbc failed east:output-different
west:output-different
> testing/pluto/ikev2-algo-06-aes-aes_xcbc passed
< testing/pluto/ikev2-algo-ike-sha2-01 failed east:output-different
west:output-different
< testing/pluto/ikev2-algo-ike-sha2-02 failed east:output-different
west:output-different
< testing/pluto/ikev2-algo-ike-sha2-03 failed east:output-different
west:output-different
> testing/pluto/ikev2-algo-ike-sha2-01 passed
> testing/pluto/ikev2-algo-ike-sha2-02 passed
> testing/pluto/ikev2-algo-ike-sha2-03 passed
< testing/pluto/ikev2-x509-23-no-ca passed
> testing/pluto/ikev2-x509-23-no-ca failed east:output-different
testing/pluto/ikev2-x509-23-no-ca/OUTPUT/east.console.diff
"ikev2-westnet-eastnet-x509-cr" #1: ERROR: Peer's Certificate issuer is not
recognized.
+"ikev2-westnet-eastnet-x509-cr" #1: ERROR: Peer's Certificate issuer is not
recognized.
< testing/pluto/newoe-21-liveness-clear passed
> testing/pluto/newoe-21-liveness-clear failed east:output-different
testing/pluto/newoe-21-liveness-clear/OUTPUT/east.console.diff
-"private-or-clear#192.1.3.0/24"[1] ...192.1.3.209 #2: liveness_check - peer
192.1.3.209 has not responded in 12 seconds, with a timeout of 10, taking
action:clear
+"private-or-clear#192.1.3.0/24"[1] ...192.1.3.209 #2: liveness_check - peer
192.1.3.209 has not responded in 11 seconds, with a timeout of 10, taking
action:clear
< testing/pluto/newoe-27-replace-sa-auth-authnull passed
> testing/pluto/newoe-27-replace-sa-auth-authnull failed road:output-different
testing/pluto/newoe-27-replace-sa-auth-authnull/OUTPUT/road.console.diff
-000 #1: "private-or-clear#192.1.2.0/24"[1] ...192.1.2.23:500 STATE_PARENT_I2
(sent v2I2, expected v2R2); EVENT_SA_REKEY in XXs; idle;
+000 #1: "private-or-clear#192.1.2.0/24"[1] ...192.1.2.23:500 STATE_PARENT_I2
(sent v2I2, expected v2R2); EVENT_SA_REPLACE in XXs; idle;
< testing/pluto/newoe-18-poc-cop-port22-transport failed road:output-different
> testing/pluto/newoe-18-poc-cop-port22-transport passed
< testing/pluto/ikev2-liveness-09 failed west:output-different
> testing/pluto/ikev2-liveness-09 passed
< testing/pluto/ikev2-child-03-dh-none-pfs-no failed west:output-different
< testing/pluto/ikev2-child-04-dh-none-pfs-yes failed west:output-different
> testing/pluto/ikev2-child-03-dh-none-pfs-no passed
> testing/pluto/ikev2-child-04-dh-none-pfs-yes passed
< testing/pluto/xauth-pluto-14 passed
> testing/pluto/xauth-pluto-14 failed road:output-different
testing/pluto/xauth-pluto-14/OUTPUT/road.console.diff
-006 #2: "modecfg-road-east", username=use3, type=ESP, add_time=1234567890,
inBytes=336, outBytes=336
+006 #2: "modecfg-road-east", username=use3, type=ESP, add_time=1234567890,
inBytes=716, outBytes=604
< testing/pluto/xauth-pluto-16 failed road:output-different
> testing/pluto/xauth-pluto-16 passed
< testing/pluto/xauth-pluto-26 passed
> testing/pluto/xauth-pluto-26 failed road:output-different
testing/pluto/xauth-pluto-26/OUTPUT/road.console.diff
-4 packets transmitted, 4 received, 0% packet loss, time XXXX
+4 packets transmitted, 3 received, 25% packet loss, time XXXX
> testing/pluto/interop-ikev1-strongswan-01-xauth failed east:output-different
> west:output-different
testing/pluto/interop-ikev1-strongswan-01-xauth/OUTPUT/east.console.diff
+strongswan 5.7.1 must be installed
< testing/pluto/l2tp-02 failed north:output-different
> testing/pluto/l2tp-02 passed
< testing/pluto/nss-cert-nosecret passed
> testing/pluto/nss-cert-nosecret failed east:output-different
> west:output-different
testing/pluto/nss-cert-nosecret/OUTPUT/east.console.diff
-000 End certificate "east" - SN: 0xXXXX
+000 End certificate "east" - SN: 0xXX
and many more
< testing/pluto/nss-cert-ocsp-02 passed
> testing/pluto/nss-cert-ocsp-02 failed east:output-different
testing/pluto/nss-cert-ocsp-02/OUTPUT/east.console.diff
"nss-cert-ocsp" #1: certificate revoked!
+"nss-cert-ocsp" #1: certificate revoked!
< testing/pluto/nss-cert-ocsp-02-ikev2 passed
> testing/pluto/nss-cert-ocsp-02-ikev2 failed east:output-different
testing/pluto/nss-cert-ocsp-02-ikev2/OUTPUT/east.console.diff
"nss-cert-ocsp" #1: certificate revoked!
+"nss-cert-ocsp" #1: certificate revoked!
< testing/pluto/nss-cert-ocsp-03-strict passed
> testing/pluto/nss-cert-ocsp-03-strict failed west:output-different
testing/pluto/nss-cert-ocsp-03-strict/OUTPUT/west.console.diff
1v1 "nss-cert-ocsp" #1: STATE_MAIN_I3: sent MI3, expecting MR3
-003 "nss-cert-ocsp" #1: ignoring informational payload INVALID_ID_INFORMATION,
msgid=00000000, length=12
-003 "nss-cert-ocsp" #1: received and ignored notification payload:
INVALID_ID_INFORMATION
< testing/pluto/nss-cert-ocsp-07-nourl passed
> testing/pluto/nss-cert-ocsp-07-nourl failed east:output-different
testing/pluto/nss-cert-ocsp-07-nourl/OUTPUT/east.console.diff
"nss-cert-ocsp" #1: certificate revoked!
+"nss-cert-ocsp" #1: certificate revoked!
< testing/pluto/nss-cert-ocsp-08-post passed
> testing/pluto/nss-cert-ocsp-08-post failed east:output-different
testing/pluto/nss-cert-ocsp-08-post/OUTPUT/east.console.diff
grep -i "certificate revoked" /tmp/pluto.log
"nss-cert-ocsp" #1: certificate revoked!
+"nss-cert-ocsp" #1: certificate revoked!
east #
grep ERROR /tmp/pluto.log
+"nss-cert-ocsp" #1: ERROR: Peer's Certificate has been revoked.
"nss-cert-ocsp" #1: ERROR: Peer's Certificate has been revoked.
< testing/pluto/nss-cert-badca-01 passed
> testing/pluto/nss-cert-badca-01 failed east:output-different
testing/pluto/nss-cert-badca-01/OUTPUT/east.console.diff
"nss-cert" #1: ERROR: Issuer certificate is invalid.
+"nss-cert" #1: ERROR: Issuer certificate is invalid.
_______________________________________________
Swan-dev mailing list
Swan-dev@lists.libreswan.org
https://lists.libreswan.org/mailman/listinfo/swan-dev
