Hi!
Does anyone know how to use nss3.8 pkcs11 module to perform public key
encrypt via hardware?
If you are referring to the software security device, known as softokn3.dll, this PKCS#11 module only works in software, not hardware.
In order to perform encryption in hardware, you must have a PKCS#11 module for your hardware device. You would perform all the operations similarly as you would in the software device, except that the slot pointer you pass would be a slot of your hardware device, rather than the PK11_InternalKeySlot() or PK11_InternalSlot(). From your code snippets, it appears you may already have that PKCS#11 module for your device available.
What is the application for which you are you trying to use your hardware public key encryption for ? The way to use the hardware key differs whether you are trying to use your key in hardware for SSL server-auth or SSL client-auth, or if you want to do a low-level RSA encrypt for a proprietary application.
In the later case, you could use the functions PK11_PubEncryptRaw / PK11_PubDecryptRaw.
