Trying to establish an ipsec tunnel to a debian linux box with openswan, using this entry in ipsec.conf:
ike active esp from 192.168.1.0/24 to 192.168.2.0/24 peer a.b.c.d srcid "[EMAIL PROTECTED]" dstid "[EMAIL PROTECTED]" psk xxxxxxxxxxx I get 'PAYLOAD MALFORMED' in the middle of the phase 1 negotiation: After the transforms are agreed upon and the nonces are exchanged, the message containing the ID payload is rejected by openBSD, either with a notification 'PAYLOAD MALFORMED' or with notification 'INVALID PAYLOAD TYPE' Here is a snippet of isakmpd.pcap: 21:38:55.438591 a.b.c.d.500 > u.v.w.x.500: [udp sum ok] isakmp v1.0 exchange ID_PROT cookie: 251068307c823c51->5086ce0f33dfbb37 msgid: 00000000 len: 92 payload: ID len: 9336 [|isakmp] [ttl 0] (id 1, len 120) 21:38:55.439228 u.v.w.x.500 > a.b.c.d.500: [udp sum ok] isakmp v1.0 exchange INFO cookie: 88fba1fcd13186bd->0000000000000000 msgid: 00000000 len: 40 payload: NOTIFICATION len: 12 notification: INVALID PAYLOAD TYPE [ttl 0] (id 1, len 68) where a.b.c.d is openswan, and u.v.w.x is openbsd. The IDs are of type USER_FQDN ( or should be, at least ). The len field in the received packet seems queer. Maybe this causes the problem. This error only occurs, when the phase-1 exchange is initiated by openswan. If openbsd starts the phase-1 exchange, all seems ok. I would think this is an openswan problem, but how can I prove this? I have no access to the openswan box. Can I get more information about the offending packet, like a decrypted hexdump or else? Any hints are welcome. Regards Christoph