Problem solved. And many thanks Sebastian for pointing to the right direction. In fact I was well aware that Keyper uses the keymap.db for key mapping. The default location which can't be changed, at least I failed to find a way to change it is /root/Keyper/PKCS11Provider/keymap.db. I'm running both signer and enforcer as user opendnssec with a different home directory (/usr/local/ods), so as a fix I moved /root/Keyper to /usr/local/ods/Keyper and created a link in /root with name Keyper pointing to /usr/local/ods/Keyper and then all commands worked both as user opendnssec and user root. Month or two later I decided to separate the opendnssec binaries and data and moved the Keyper data to /ods-data/Keyper. The new setup continued using the same keys and it still worked well. The the problems started when I decided wipe the data and keys and the the signer failed to sign the zone because it was looking for the mapping of the keys at the old location /usr/local/ods/Keyper. The fix was to change the home directory for user opendnssec. Thank you again.
Emil On Wed, Dec 17, 2014 at 4:15 AM, Sebastian Castro <[email protected]> wrote: > > > > On 17/12/14 12:56 am, Emil Natan wrote: > > Hi Matthijs and thank you for your reply. > > > > Hi Emil: > > Your problem seems really odd, but for some reason not strange. We've > done some testing with the AEP Keyper, and it seems there is a mapping > between key id and HSM used that lives in a BerkeleyDB file somewhere in > the file system. > > I don't recall the location of the file at the moment, and don't have > notes, but came across with something similar before. > > You can find where the file is while stracing the command > > ods-hsmutil generate Keyper rsa 1024 > > Also you can try with ods-hsmutil to generate a DNSKEY from an existing > key, perhaps the problem is your program doesn't have access to read the > mapping file. > > If you run > > ods-hsmutil dnskey 39a954b0fccb0f5ed73614d5fc1a8144 test. > > as the root used should work, but if you run > > sudo -u opendnssec ods-hsmutil dnskey 39a954b0fccb0f5ed73614d5fc1a8144 > test. > > it should fail. > > Let us know how it works, I'll ask internally to find out if someone > remembers the name of the bloody file! > > > Here is how it goes for me. > > > > I start with: > > Zone: Keytype: State: Date of next > > transition: > > XXX KSK active 2016-01-16 > > 09:49:45 > > XXX ZSK active 2015-04-18 > 22:40:55 > > > > root@debugsigner002:~# ods-hsmutil purge Keyper > > Purging all keys from repository: Keyper > > 12 keys found. > > > > Are you sure you want to remove ALL keys from repository Keyper ? > > (YES/NO) yes > > > > Starting purge... > > Key remove successful: fdd17d120d3e548a104dda856d84c770 > > ... > > Key remove successful: db97ded0cc231c3908f8f20f5ce21229 > > Key remove successful: f81e4b2cb33eec780320b6ceeb6f6bb8 > > Purge done. > > > > root@debugsigner002:~# /opt/Keyper/PKCS11Provider/inittoken > > ... > > PKCS11 Slot : 0 > > PKCS11 Label : aepkeyper > > Keyper Model : Keyper Ent 1126 > > Keyper Serial : > > Keyper version : 2.0 > > App : 020 > > ABL : 029 > > AL : 02 > > -------------------------------------------- > > Token initialised OK > > ******************************************** > > > > To remove the zone I actually comment it out from zonelist.xml, then: > > > > root@debugsigner002:~# ods-ksmutil update zonelist > > zonelist filename set to /ods-data/etc/opendnssec/zonelist.xml. > > kasp filename set to /ods-data/etc/opendnssec/kasp.xml. > > Removing zone XXX from database > > Notifying enforcer of new database... > > > > I stopped both ODS daemons. > > > > root@debugsigner002:~# ps auxww | grep ods > > root 14452 0.0 0.0 11744 896 pts/2 S+ 13:31 0:00 grep > > --color=auto ods > > > > Initialize ODS, all the warnings are skipped, but no errors. > > > > root@debugsigner002:~# ods-ksmutil setup > > > > *WARNING* This will erase all data in the database; are you sure? [y/N] y > > zonelist filename set to /ods-data/etc/opendnssec/zonelist.xml. > > kasp filename set to /ods-data/etc/opendnssec/kasp.xml. > > Repository Keyper found > > No Maximum Capacity set. > > RequireBackup set. > > INFO: The XML in /ods-data/etc/opendnssec/conf.xml is valid > > INFO: The XML in /ods-data/etc/opendnssec/zonelist.xml is valid > > INFO: The XML in /ods-data/etc/opendnssec/kasp.xml is valid > > Policy XXXTLD found > > > > Generate new keys. > > > > root@debugsigner002:~# ods-ksmutil key generate --policy XXXTLD > > --zonetotal 1 --interval P2Y > > Key sharing is Off > > Info: converting P2Y to seconds; M interpreted as 31 days, Y interpreted > > as 365 days > > HSM opened successfully. > > Info: 0 zone(s) found on policy "XXXTLD" > > Info: Keys will actually be generated for a total of 1 zone(s) as > > specified by zone total parameter > > 2 new KSK(s) (2048 bits) need to be created for policy XXXTLD: > > keys_to_generate(2) = keys_needed(2) - keys_available(0). > > 6 new ZSK(s) (1024 bits) need to be created for policy XXXTLD: > > keys_to_generate(6) = keys_needed(6) - keys_available(0). > > *WARNING* This will create 2 KSKs (2048 bits) and 6 ZSKs (1024 bits) > > Are you sure? [y/N] > > y > > Created KSK size: 2048, alg: 8 with id: 39a954b0fccb0f5ed73614d5fc1a8144 > > in repository: Keyper and database. > > Created KSK size: 2048, alg: 8 with id: 47dc08d7c5be2104b18a9f7a1702e6b0 > > in repository: Keyper and database. > > Created ZSK size: 1024, alg: 8 with id: 64504804f1dc34cd44fa83cbede95275 > > in repository: Keyper and database. > > Created ZSK size: 1024, alg: 8 with id: ec77d359ccdde3e38b222423a5d2075f > > in repository: Keyper and database. > > Created ZSK size: 1024, alg: 8 with id: 669a0a563fa03c62fc58d20e85628b35 > > in repository: Keyper and database. > > Created ZSK size: 1024, alg: 8 with id: e799a40efda79c8e98a76adc72470f6d > > in repository: Keyper and database. > > Created ZSK size: 1024, alg: 8 with id: 0618eb27e1061e37df6bf6a055c85160 > > in repository: Keyper and database. > > Created ZSK size: 1024, alg: 8 with id: 424fbb66fbaf3605b4d935b473d1be01 > > in repository: Keyper and database. > > NOTE: keys generated in repository Keyper will not become active until > > they have been backed up > > all done! hsm_close result: 0 > > > > I also mark the keys as backed up. > > > > root@debugsigner002:~# ods-ksmutil backup prepare > > Marked all repositories as pre-backed up at 2014-12-16 13:40:15 > > root@debugsigner002:~# ods-ksmutil backup commit > > Marked all repositories as backed up at 2014-12-16 13:40:21 > > > > This time I stopped the signer and enforcer before setup, so I start > them. > > > > root@debugsigner002:~# ps auxww | grep ods > > opendns+ 14492 0.0 0.5 128840 5548 ? Ss 13:42 0:00 > > /ods-bin/sbin/ods-enforcerd > > opendns+ 14501 0.0 0.6 533744 7068 ? Ssl 13:42 0:00 > > /ods-bin/sbin/ods-signerd > > root 14514 0.0 0.0 11744 896 pts/1 S+ 13:45 0:00 grep > > --color=auto ods > > > > I added the zone, again by editing zonelist.xml and ... > > > > root@debugsigner002:~# ods-ksmutil update zonelist > > zonelist filename set to /ods-data/etc/opendnssec/zonelist.xml. > > kasp filename set to /ods-data/etc/opendnssec/kasp.xml. > > Zone XXX found; policy set to XXXTLD > > Notifying enforcer of new database... > > > > And I end up with the same problem. > > > > Dec 16 13:46:08 debugsigner002 ods-signerd: [hsm] libhsm connection ok > > Dec 16 13:46:08 debugsigner002 ods-signerd: [hsm] unable to get key: key > > 39a954b0fccb0f5ed73614d5fc1a8144 not found > > Dec 16 13:46:08 debugsigner002 ods-signerd: [zone] unable to publish > > dnskeys for zone XXX: error creating dnskey > > Dec 16 13:46:08 debugsigner002 ods-signerd: [tools] unable to read zone > > XXX: failed to publish dnskeys (General error) > > Dec 16 13:46:08 debugsigner002 ods-signerd: [worker[1]] CRITICAL: failed > > to sign zone XXX: General error > > > > And ods-ksmutil can still list the keys: > > > > root@debugsigner002:~# ods-ksmutil key list -v > > Zone: Keytype: State: Date of next > > transition (to): Size: Algorithm: CKA_ID: > > Repository: Keytag: > > XXX ZSK active 2015-04-19 > > 13:46:07 (retire) 1024 8 64504804f1dc34cd44fa83cbede95275 > > Keyper 5680 > > XXX KSK publish 2014-12-16 > > 17:51:07 (ready) 2048 8 39a954b0fccb0f5ed73614d5fc1a8144 > > Keyper 6962 > > > > I'll send you the full log off-list. > > Thanks again. > > > > Emil > > > > On Tue, Dec 16, 2014 at 12:18 PM, Matthijs Mekking > > <[email protected] <mailto:[email protected]>> wrote: > > > > Hi Emil, > > > > Short: I tried to simulate your use case (with SoftHSM, on > > ubuntu-trusty-64 VM), but it seems to work for me. Perhaps I used > > slightly different commands? Can you share your used commands? > > > > Best regards, > > Matthijs > > > > > > Audit trail: > > > > I started with Keys: > > Zone: Keytype: State: Date of next > transition: > > example.com <http://example.com> KSK publish > > 2014-12-16 23:55:02 > > example.com <http://example.com> ZSK active > > 2015-03-16 09:55:02 > > > > On 16-12-14 08:54, Emil Natan wrote: > > > Good morning, > > > > > > I have a test environment with ODS 1.4.6 and Keyper HSM where > signing > > > zones was working until I decided to remove all keys and start > from scratch. > > > I removed all keys with "ods-hsmutil purge"\ > > > > $ sudo ods-hsmutil purge SoftHSM > > Purging all keys from repository: SoftHSM > > 2 keys found. > > > > Are you sure you want to remove ALL keys from repository SoftHSM ? > > (YES/NO) YES > > > > Starting purge... > > Key remove successful: 816416e1255a1724021895b531c0e313 > > Key remove successful: 615ef6c218cc6bc6d714a0742a07617b > > Purge done. > > > > > > > reinitialized the HSM\ > > > > Don't think this is necessary, but okay: > > > > $ sudo softhsm --init-token --slot 0 --label "OpenDNSSEC" > > The SO PIN must have a length between 4 and 255 characters. > > Enter SO PIN: > > The user PIN must have a length between 4 and 255 characters. > > Enter user PIN: > > The token has been initialized. > > > > > > > removed the single zone I used to sign\ > > > > $ sudo ods-ksmutil zone delete --zone example.com < > http://example.com> > > zonelist filename set to /etc/opendnssec/zonelist.xml. > > Zone list updated: 1 removed, 0 added, 0 updated. > > > > > > > reinitialized the database "ods-ksmutil setup"\ > > > > I think you should first stop the opendnssec service, but I will not > do > > that now: > > > > $ sudo ods-ksmutil setup > > *WARNING* This will erase all data in the database; are you sure? > > [y/N] y > > fixing permissions on file /var/opendnssec/kasp.db > > zonelist filename set to /etc/opendnssec/zonelist.xml. > > kasp filename set to /etc/opendnssec/kasp.xml. > > Repository SoftHSM found > > No Maximum Capacity set. > > RequireBackup NOT set; please make sure that you know the potential > > problems of using keys which are not recoverable > > INFO: The XML in /etc/opendnssec/conf.xml is valid > > INFO: The XML in /etc/opendnssec/zonelist.xml is valid > > INFO: The XML in /etc/opendnssec/kasp.xml is valid > > WARNING: In policy default, Y used in duration field for Keys/KSK > > Lifetime (P1Y) in /etc/opendnssec/kasp.xml - this will be > interpreted as > > 365 days > > WARNING: In policy lab, Y used in duration field for Keys/KSK > Lifetime > > (P1Y) in /etc/opendnssec/kasp.xml - this will be interpreted as 365 > days > > Policy default found > > Info: converting P1Y to seconds; M interpreted as 31 days, Y > interpreted > > as 365 days > > Policy lab found > > Info: converting P1Y to seconds; M interpreted as 31 days, Y > interpreted > > as 365 days > > > > > > > pregenerated new keys\ > > > > But you have no zones currently (you removed the single zone)? > > > > $ sudo ods-ksmutil key generate --policy default --interval P1Y > > Key sharing is Off > > Info: converting P1Y to seconds; M interpreted as 31 days, Y > interpreted > > as 365 days > > HSM opened successfully. > > Info: 0 zone(s) found on policy "default" > > No zones on policy default, skipping... > > > > > > > added a zone\ > > > > $ sudo ods-ksmutil zone add --zone example.com <http://example.com> > > zonelist filename set to /etc/opendnssec/zonelist.xml. > > Imported zone: example.com <http://example.com> > > > > > > > updated, restarted all services. > > > > $ sudo ods-control stop > > Stopping enforcer... > > Stopping signer engine... > > Engine shut down. > > > > $ sudo ods-control start > > Starting enforcer... > > OpenDNSSEC ods-enforcerd started (version 1.4.6), pid 28343 > > Starting signer engine... > > OpenDNSSEC signer engine version 1.4.6 > > Engine running. > > > > > Everything seems to worked well, but the signer does not find one > of the > > > keys to sign the zone, more specifically the KSK. I went the above > > > process few times, always ending with: > > > > > > Dec 16 09:40:27 debugsigner002 ods-signerd: [hsm] unable to get > key: key > > > f81e4b2cb33eec780320b6ceeb6f6bb8 not found > > > Dec 16 09:40:27 debugsigner002 ods-signerd: [zone] unable to > publish > > > dnskeys for zone XXX: error creating dnskey > > > Dec 16 09:40:27 debugsigner002 ods-signerd: [tools] unable to read > zone > > > XXX: failed to publish dnskeys (General error) > > > Dec 16 09:40:27 debugsigner002 ods-signerd: [worker[4]] CRITICAL: > failed > > > to sign zone XXX: General error > > > > For me, it finds the old key in the > > `/var/opendnssec/tmp/example.com.backup2` file and decides it is > > corrupted: > > > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [hsm] libhsm > > connection opened succesfully > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [engine] signer > > started (version 1.4.6), pid 28355 > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [hsm] unable to > > get key: key 615ef6c218cc6bc6d714a0742a07617b not found > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [zone] unable > to > > publish dnskeys for zone example.com <http://example.com>: error > > creating dnskey > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [zone] > corrupted > > backup file zone example.com <http://example.com>: unable to publish > > dnskeys (General error) > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [engine] > unable to > > recover zone example.com <http://example.com> from backup, > > performing full sign > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [signconf] zone > > example.com <http://example.com> signconf: RESIGN[PT7200S] > > REFRESH[PT259200S] > > VALIDITY[PT1209600S] DENIAL[PT1209600S] JITTER[PT43200S] > OFFSET[PT3600S] > > NSEC[50] DNSKEYTTL[PT3600S] SOATTL[PT3600S] MINIMUM[PT3600S] > > SERIAL[unixtime] > > Dec 16 10:07:59 vagrant-ubuntu-trusty-64 ods-signerd: [STATS] > > example.com <http://example.com> 1418724479 RR[count=61 time=0(sec)] > > NSEC3[count=60 > > time=0(sec)] RRSIG[new=112 reused=0 time=0(sec) avg=0(sig/sec)] > > TOTAL[time=0(sec)] > > > > > > > The key exist in both HSM and database. ods-hsmutil lists it: > > > > > > root@debugsigner002:~# ods-hsmutil list | grep > > > f81e4b2cb33eec780320b6ceeb6f6bb8 > > > Keyper f81e4b2cb33eec780320b6ceeb6f6bb8 RSA/2048 > > > > > > ods-ksmutil shows it: > > > > > > root@debugsigner002:~# ods-ksmutil key list -v > > > Keys: > > > Zone: Keytype: State: Date of > next > > > transition (to): Size: Algorithm: CKA_ID: > > > Repository: Keytag: > > > XXX KSK active 2016-01-16 > > > 09:49:45 (retire) 2048 8 > f81e4b2cb33eec780320b6ceeb6f6bb8 > > > Keyper 6061 > > > XXX ZSK active 2015-04-18 > > > 22:40:55 (retire) 1024 8 > d2aa0ba9af0f41429d23ea387abb836a > > > Keyper > > > > > > external tools - dnssec-keyfromlabel can use it. > > > No other errors in the log. > > > > > > Any ideas what's wrong? Suggestions what else to try? > > > Thanks. > > > > > > Emil > > > > > > > > > > > > > > > > > > _______________________________________________ > > > Opendnssec-user mailing list > > > [email protected] > > <mailto:[email protected]> > > > https://lists.opendnssec.org/mailman/listinfo/opendnssec-user > > > > > > > _______________________________________________ > > Opendnssec-user mailing list > > [email protected] > > <mailto:[email protected]> > > https://lists.opendnssec.org/mailman/listinfo/opendnssec-user > > > > > > > > _______________________________________________ > > Opendnssec-user mailing list > > [email protected] > > https://lists.opendnssec.org/mailman/listinfo/opendnssec-user > > > > -- > Sebastian Castro > Technical Research Manager > .nz Registry Services (New Zealand Domain Name Registry Limited) > desk: +64 4 495 2337 > mobile: +64 21 400535 >
_______________________________________________ Opendnssec-user mailing list [email protected] https://lists.opendnssec.org/mailman/listinfo/opendnssec-user
