Hi, I have an issue related to RSA decryption while using https. I have setup a dummy https server and captured packets on wireshark.
As per the RFC - in client key exchange message the premaster is encrypted using Server's public key and sent to server. So, I have captured the encrypted premaster and tried decrypting it using server's private key using RSA algorithm. But below is some strange error I encountered - the RSA decryption failed with '-1' return code, but while interpreting the error code - it shows no error.
[root@killbill performance_test]# ./decrypt RSA_private_decrypt() failed : -1 Decrypt failed: error:00000000:lib(0):func(0):reason(0)Could someone please advise if I am doing anything wrong? Attaching the server.key (private key), test trace and test C code for reference.
Any pointers will be appreciated.P.S. I am not quite sure if this query belongs to this list. Please redirect me to correct list, if so.
-- Thanks, Nilesh
#include <openssl/rsa.h>
#include <openssl/aes.h>
#include <openssl/md5.h>
#include <openssl/sha.h>
#include <openssl/err.h>
#include <string.h>
#include <time.h>
#include <sys/time.h>
#include "digest.h"
#include <errno.h>
unsigned char to_decrypt_key[1024] = "\x00\x80\xb1\x75\xe4\xa5\x0d\xf0\xd6\x63\xdc\x17\x87\x5b\xbd\xc0\x43\xe2\x48\x87\x90\x01\xa8\xd3\x2d\x62\x0e\x4a\x61\x4a\x03\x57\x2f\x03\xfc\xb6\x24\xe4\x27\xde\xb7\xac\xf8\xa5\xaf\x6b\x13\x4e\x1c\x49\x02\x5d\xa3\x50\x21\xb4\x81\xd6\xc2\x17\x8c\x83\x0d\x0b\xd4\xc5\xb0\x12\x8d\x40\xd6\x70\x44\xa2\x6b\x1a\xa2\x17\x66\xde\x71\xcc\x2d\xb2\xbd\xe9\xa7\x83\x9c\xb4\x00\x20\xfb\xcc\x80\x5e\x15\x83\xea\x17\xbb\x53\xfd\x8f\xec\x31\xdb\xf6\x62\x57\xbe\x12\x89\x26\x4c\x86\x8c\xc4\xbe\xd4\x6f\xe1\xe8\x77\xe7\xc0\xc0\x40";
int main(void)
{
RSA *rsa;
FILE *fp;
int i, check, pre_master_len = 128;
unsigned char pre_master[1024];
char err_buf[240];
rsa = RSA_new();
/* 1 Open server's private key file */
if((fp = fopen("server.key", "rb")) == NULL) {
printf("Cannot open server key file.\n");
return;
}
/* 2 Generate RSA struct from private key file */
PEM_read_RSAPrivateKey(fp, &rsa, NULL, NULL);
/* 3 Check for successful key generation */
if(RSA_check_key(rsa) != 1) {
printf("RSA_check_key(): PrivateKey check failed\n");
return;
}
/* 4 Using Private RSA Key, decode the client_pre_master_secret */
check = RSA_private_decrypt(pre_master_len, to_decrypt_key,
pre_master, rsa, RSA_PKCS1_PADDING);
if(check == -1) {
printf("RSA_private_decrypt() failed : %d\n", check);
printf("Decrypt failed: %s",
ERR_error_string(ERR_get_error(), err_buf));
exit(1);
}
return 0;
}
https_trace.pcap
Description: application/cap
server.key
Description: application/pgp-keys
