Re: OpenSSL hash memory leak
If it helps, sometimes I do get the following errors for the same and subsequent reboot: Alignment trap: sh (601) PC=0xb6e008f8 Instr=0x4589c0d7 Address=0x00d7 FSR 0x801 Alignment trap: login (584) PC=0xb6e6ab00 Instr=0xe5951000 Address=0xd27cdc63 FSR 0x001 Kernel panic - not syncing: Attempted to kill init! exitcode=0x000b On Sun, 24 Feb 2019 at 15:58, prithiraj das wrote: > Hi All, > > Apart from my code posted in this mailchain, I tried testing using the > OpenSSL commands. I ran *openssl dgst -sha256 Test_blob.* Test_blob and > all files mentioned below are almost 44 MB (or more). > > The first time buff/cache value increased by 44MB (size of the file) > *totalused freeshared > buff/cache available* > *Mem: 252180 12984 181544 284 57652 >231188* > *Swap: 0 0 0* > > I ran the same OpenSSL command again with the same file, and the result of > free command remained the same > *totalused freeshared > buff/cache available* > *Mem: 252180 12984 181544 284 57652 > 231188* > *Swap: 0 0 0* > > Next I ran the same command with a different file (let's say Test_blob2) > and ran the free command after it, result:- >* totalusedfrees**hared > buff/cache available* > *Mem:252180 12948 137916 284 101316 >231200* > *Swap:0 0 0* > > The *buff/cache* value has increased by the size of the file concerned* > (almost > 44MB)* > If I run the same command the 3rd time with another file not previously > used (let's say Test_blob3), the following happens > > *Kernel panic - not syncing: Attempted to kill init! exitcode=0x000b* > *Rebooting in 15 seconds..* > > Is there a way to resolve this problem, How do I clear the buff/cache? > > On Sun, 24 Feb 2019 at 03:15, Georg Höllrigl > wrote: > >> Hello, >> >> >> >> I guess you’re not seeing a memory leak, but just normal behaviour of >> linux file system cache. >> >> Buff/cache is keeping files in memory that were least accessed as long as >> not needed by other stuff. >> >> You don’t need to free the buffer/cache, because linux does that >> automatically, when memory is needed. >> >> >> >> Kind Regards, >> >> Georg >> >> >> >> *Von:* openssl-users *Im Auftrag von >> *prithiraj das >> *Gesendet:* 23 February 2019 18:25 >> *An:* Jordan Brown >> *Cc:* openssl-users@openssl.org >> *Betreff:* Re: OpenSSL hash memory leak >> >> >> >> Hi, >> >> This is how I have initialized my variables:- >> >> >> >> EVP_MD_CTX *mdctx; >> >> const EVP_MD *md; >> >> int i; >> >> HASH hash_data; >> >> unsigned char message_data[BUFFER_SIZE]; >> >> >> >> BUFFER_SIZE has been defined as 131072 >> >> and HASH is my hash structure defined to hold the message digest, message >> digest length and message digest type >> >> >> >> On Sat, 23 Feb 2019 at 00:17, Jordan Brown >> wrote: >> >> The most obvious question is "how are you allocating your message_data >> buffer?". You don't show that. >> >> >> >> On 2/22/2019 2:27 AM, prithiraj das wrote: >> >> >> >> Hi All, >> >> >> >> Using OpenSSL 1.0.2g, I have written a code to generate the hash of a >> file in an embeddded device having linux OS and low memory capacity and the >> files are generally of size 44 MB or more. The first time or even the >> second time on some occasions, the hash of any file is successfully >> generated. On the 3rd or 4th time (possibly due to lack of memory/memory >> leak), the system reboots before the hash can be generated. After restart, >> the same thing happens when the previous steps are repeated. >> >> The stats below shows the memory usage before and after computing the >> hash. >> >> >> >> *root@at91sam9m10g45ek:~# free* >> >> * totalused free shared >> buff/cache available* >> >> *Mem: 252180 13272 223048 280 15860 >> 230924* >> >> *Swap:0 0 0* >> >> >> >> *After computing hash :-* >> >> *root
Re: OpenSSL hash memory leak
Hi All,
Apart from my code posted in this mailchain, I tried testing using the
OpenSSL commands. I ran *openssl dgst -sha256 Test_blob.* Test_blob and all
files mentioned below are almost 44 MB (or more).
The first time buff/cache value increased by 44MB (size of the file)
*totalused freeshared
buff/cache available*
*Mem: 252180 12984 181544 284 57652
231188*
*Swap: 0 0 0*
I ran the same OpenSSL command again with the same file, and the result of
free command remained the same
*totalused freeshared
buff/cache available*
*Mem: 252180 12984 181544 284 57652
231188*
*Swap: 0 0 0*
Next I ran the same command with a different file (let's say Test_blob2)
and ran the free command after it, result:-
* totalusedfrees**hared
buff/cache available*
*Mem:252180 12948 137916 284 101316
231200*
*Swap:0 0 0*
The *buff/cache* value has increased by the size of the file concerned* (almost
44MB)*
If I run the same command the 3rd time with another file not previously
used (let's say Test_blob3), the following happens
*Kernel panic - not syncing: Attempted to kill init! exitcode=0x000b*
*Rebooting in 15 seconds..*
Is there a way to resolve this problem, How do I clear the buff/cache?
On Sun, 24 Feb 2019 at 03:15, Georg Höllrigl wrote:
> Hello,
>
>
>
> I guess you’re not seeing a memory leak, but just normal behaviour of
> linux file system cache.
>
> Buff/cache is keeping files in memory that were least accessed as long as
> not needed by other stuff.
>
> You don’t need to free the buffer/cache, because linux does that
> automatically, when memory is needed.
>
>
>
> Kind Regards,
>
> Georg
>
>
>
> *Von:* openssl-users *Im Auftrag von
> *prithiraj
> das
> *Gesendet:* 23 February 2019 18:25
> *An:* Jordan Brown
> *Cc:* openssl-users@openssl.org
> *Betreff:* Re: OpenSSL hash memory leak
>
>
>
> Hi,
>
> This is how I have initialized my variables:-
>
>
>
> EVP_MD_CTX *mdctx;
>
> const EVP_MD *md;
>
> int i;
>
> HASH hash_data;
>
> unsigned char message_data[BUFFER_SIZE];
>
>
>
> BUFFER_SIZE has been defined as 131072
>
> and HASH is my hash structure defined to hold the message digest, message
> digest length and message digest type
>
>
>
> On Sat, 23 Feb 2019 at 00:17, Jordan Brown
> wrote:
>
> The most obvious question is "how are you allocating your message_data
> buffer?". You don't show that.
>
>
>
> On 2/22/2019 2:27 AM, prithiraj das wrote:
>
>
>
> Hi All,
>
>
>
> Using OpenSSL 1.0.2g, I have written a code to generate the hash of a file
> in an embeddded device having linux OS and low memory capacity and the
> files are generally of size 44 MB or more. The first time or even the
> second time on some occasions, the hash of any file is successfully
> generated. On the 3rd or 4th time (possibly due to lack of memory/memory
> leak), the system reboots before the hash can be generated. After restart,
> the same thing happens when the previous steps are repeated.
>
> The stats below shows the memory usage before and after computing the
> hash.
>
>
>
> *root@at91sam9m10g45ek:~# free*
>
> * totalused free shared
> buff/cache available*
>
> *Mem: 252180 13272 223048 280 15860
> 230924*
>
> *Swap:0 0 0*
>
>
>
> *After computing hash :-*
>
> *root@at91sam9m10g45ek:~# free*
>
> * totalused free shared
> buff/cache available*
>
> *Mem: 252180 13308 179308280 59564
>230868*
>
> *Swap: 00 0*
>
>
>
> Buff/cache increases by almost 44MB (same as file size) everytime I
> generate the hash and free decreases. I believe the file is being loaded
> into buffer and not being freed.
>
>
>
> I am using the below code to compute the message digest. This code is part
> of a function ComputeHash and the file pointer here is fph.
>
>
>
> * EVP_add_digest(EVP_sha256());*
>
> * md = EVP_get_digestbyname("sha256");*
>
>
>
> * if(!md) {*
>
> *printf("Unknown message digest \n");*
>
> *exit(1);*
>
> * }*
>
> * printf("Message digest algorithm successfully loaded\n"
Re: OpenSSL hash memory leak
Hi,
This is how I have initialized my variables:-
EVP_MD_CTX *mdctx;
const EVP_MD *md;
int i;
HASH hash_data;
unsigned char message_data[BUFFER_SIZE];
BUFFER_SIZE has been defined as 131072
and HASH is my hash structure defined to hold the message digest, message
digest length and message digest type
On Sat, 23 Feb 2019 at 00:17, Jordan Brown
wrote:
> The most obvious question is "how are you allocating your message_data
> buffer?". You don't show that.
>
> On 2/22/2019 2:27 AM, prithiraj das wrote:
>
>
> Hi All,
>
> Using OpenSSL 1.0.2g, I have written a code to generate the hash of a file
> in an embeddded device having linux OS and low memory capacity and the
> files are generally of size 44 MB or more. The first time or even the
> second time on some occasions, the hash of any file is successfully
> generated. On the 3rd or 4th time (possibly due to lack of memory/memory
> leak), the system reboots before the hash can be generated. After restart,
> the same thing happens when the previous steps are repeated.
> The stats below shows the memory usage before and after computing the
> hash.
>
> *root@at91sam9m10g45ek:~# free*
> * totalused free shared
> buff/cache available*
> *Mem: 252180 13272 223048 280 15860
> 230924*
> *Swap:0 0 0*
>
> *After computing hash :-*
> *root@at91sam9m10g45ek:~# free*
> * totalused free shared
> buff/cache available*
> *Mem: 252180 13308 179308280 59564
>230868*
> *Swap: 00 0*
>
> Buff/cache increases by almost 44MB (same as file size) everytime I
> generate the hash and free decreases. I believe the file is being loaded
> into buffer and not being freed.
>
> I am using the below code to compute the message digest. This code is part
> of a function ComputeHash and the file pointer here is fph.
>
> * EVP_add_digest(EVP_sha256());*
> * md = EVP_get_digestbyname("sha256");*
>
> * if(!md) {*
> * printf("Unknown message digest \n");*
> * exit(1);*
> * }*
> * printf("Message digest algorithm successfully loaded\n");*
> * mdctx = EVP_MD_CTX_create();*
> * EVP_DigestInit_ex(mdctx, md, NULL);*
>
> * // Reading data to array of unsigned chars *
> * long long int bytes_read = 0;*
>
> * printf("FILE size of the file to be hashed is %ld",filesize); *
>
> * //reading image file in chunks below and fph is the file pointer to the
> 44MB file*
> * while ((bytes_read = fread (message_data, 1, BUFFER_SIZE, fph)) != 0)*
> * EVP_DigestUpdate(mdctx, message_data, bytes_read);*
> * EVP_DigestFinal_ex(mdctx, hash_data.md_value, _data.md_len);*
> * printf("\n%d\n",EVP_MD_CTX_size(mdctx));*
> * printf("\n%d\n",EVP_MD_CTX_type(mdctx));*
> * hash_data.md_type=EVP_MD_CTX_type(mdctx);*
> * EVP_MD_CTX_destroy(mdctx);*
> * //fclose(fp);*
> * printf("Generated Digest is:\n ");*
> * for(i = 0; i < hash_data.md_len; i++)*
> * printf("%02x", hash_data.md_value[i]);*
> * printf("\n");*
> * EVP_cleanup();*
> * return hash_data;*
>
> In the the code below, I have done fclose(fp)
> *verify_hash=ComputeHash(fp,size1);*
> *fclose(fp);*
>
> I believe that instead of loading the entire file all at once I am reading
> the 44MB file in chunks and computing the hash using the piece of code
> below: (fph is the file pointer)
> *while ((bytes_read = fread (message_data, 1, BUFFER_SIZE, fph)) != 0)*
> * EVP_DigestUpdate(mdctx, message_data, bytes_read);*
>
> Where I am going wrong? How can I free the buff/cache after computation of
> message digest? Please suggest ways to tackle this.
>
>
> Thanks and Regards,
> Prithiraj
>
>
> --
> Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris
>
>
OpenSSL hash memory leak
Hi All,
Using OpenSSL 1.0.2g, I have written a code to generate the hash of a file
in an embeddded device having linux OS and low memory capacity and the
files are generally of size 44 MB or more. The first time or even the
second time on some occasions, the hash of any file is successfully
generated. On the 3rd or 4th time (possibly due to lack of memory/memory
leak), the system reboots before the hash can be generated. After restart,
the same thing happens when the previous steps are repeated.
The stats below shows the memory usage before and after computing the hash.
*root@at91sam9m10g45ek:~# free*
* totalused free shared
buff/cache available*
*Mem: 252180 13272 223048 280 15860
230924*
*Swap:0 0 0*
*After computing hash :-*
*root@at91sam9m10g45ek:~# free*
* totalused free shared
buff/cache available*
*Mem: 252180 13308 179308280 59564
230868*
*Swap: 00 0*
Buff/cache increases by almost 44MB (same as file size) everytime I
generate the hash and free decreases. I believe the file is being loaded
into buffer and not being freed.
I am using the below code to compute the message digest. This code is part
of a function ComputeHash and the file pointer here is fph.
* EVP_add_digest(EVP_sha256());*
* md = EVP_get_digestbyname("sha256");*
* if(!md) {*
* printf("Unknown message digest \n");*
* exit(1);*
* }*
* printf("Message digest algorithm successfully loaded\n");*
* mdctx = EVP_MD_CTX_create();*
* EVP_DigestInit_ex(mdctx, md, NULL);*
* // Reading data to array of unsigned chars *
* long long int bytes_read = 0;*
* printf("FILE size of the file to be hashed is %ld",filesize); *
* //reading image file in chunks below and fph is the file pointer to the
44MB file*
* while ((bytes_read = fread (message_data, 1, BUFFER_SIZE, fph)) != 0)*
* EVP_DigestUpdate(mdctx, message_data, bytes_read);*
* EVP_DigestFinal_ex(mdctx, hash_data.md_value, _data.md_len);*
* printf("\n%d\n",EVP_MD_CTX_size(mdctx));*
* printf("\n%d\n",EVP_MD_CTX_type(mdctx));*
* hash_data.md_type=EVP_MD_CTX_type(mdctx);*
* EVP_MD_CTX_destroy(mdctx);*
* //fclose(fp);*
* printf("Generated Digest is:\n ");*
* for(i = 0; i < hash_data.md_len; i++)*
* printf("%02x", hash_data.md_value[i]);*
* printf("\n");*
* EVP_cleanup();*
* return hash_data;*
In the the code below, I have done fclose(fp)
*verify_hash=ComputeHash(fp,size1);*
*fclose(fp);*
I believe that instead of loading the entire file all at once I am reading
the 44MB file in chunks and computing the hash using the piece of code
below: (fph is the file pointer)
*while ((bytes_read = fread (message_data, 1, BUFFER_SIZE, fph)) != 0)*
* EVP_DigestUpdate(mdctx, message_data, bytes_read);*
Where I am going wrong? How can I free the buff/cache after computation of
message digest? Please suggest ways to tackle this.
Thanks and Regards,
Prithiraj
[openssl-users] RSA Digital Signing
Hi All, Using OpenSSL, I need to implement digital signing. My approach as of now is: 1) At the sender side, generate the hash of the data using sha256. 2) Encrypt the hash of the data using RSA Private key for the purpose of signing. Send this encrypted hash and the data from Step 1 to the receiverr side. 3) At the receiver's end, Decrypt the signed data(encrypted hash) using the corresponding RSA Public key. 4) Generate hash of the data and verify the decrypted content against this hash to verify the signature I was thinking of using RSA_private_encrypt() method to do the signing and RSA_public_decrypt() method to decrypt the signed hash using the corresponding RSA public key. Would the above be a bad approach especially when it comes to using the methods mentioned ? Please recommend the methods to be used that would be better for the purpose of digital signing and verification using sha256 and RSA keys Thanks and Regards, Prithiraj -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL v1.1.1 static library size reduction
Please find the above previous mail. On Mon, 24 Dec 2018 at 19:29, Salz, Rich via openssl-users < openssl-users@openssl.org> wrote: > If all you need is RSA then you will probably find it easier to write a > makefile of your own. You will have to do multiple builds to get all the > missing pieces, such as the BN facility, the memory allocation, the error > stack, etc. > -- > openssl-users mailing list > To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users > -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] OpenSSL v1.1.1 static library size reduction
Hi All, Please accept this as a gentle reminder to the previous mail in the mailchain. And also would a custom makefile (if created for this purpose) help in this regard? Thanks and Regards, Prithiraj On Fri, 21 Dec 2018 at 06:12, prithiraj das wrote: > I am using OpenSSL 1.1.1 from OpenSSL's website and trying to build > OpenSSL on a Windows 64 bit machine using Perl 64 bit version and nasm > v2.13.03. I have used the *no-shared* option in the Perl Configure to > only build the static library and the resulting size of the > *libcrypto.lib* file is almost 19 MB. The *.exe* file generated is 3173 > KB. RSA functionality (keypair generation, encryption, decryption) is what > we all need and as per the need, the goal is to reduce *libcrypto.lib *to > less than 3 MB. Using the generated .exe file is not an option. > Please suggest ways to reduce the libcrypto.lib size to less than 3 MB on > this 64 bit machine keeping only RSA functionality. > And, is it possible by any chance that the size of libcrypto.lib will be > smaller if OpenSSL is being built on a Windows 32 bit machine using a > Windows 32 bit configuration option VC-WIN32? > > Thanks and Regards, > Prithiraj > -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
[openssl-users] OpenSSL v1.1.1 static library size reduction
I am using OpenSSL 1.1.1 from OpenSSL's website and trying to build OpenSSL on a Windows 64 bit machine using Perl 64 bit version and nasm v2.13.03. I have used the *no-shared* option in the Perl Configure to only build the static library and the resulting size of the *libcrypto.lib* file is almost 19 MB. The *.exe* file generated is 3173 KB. RSA functionality (keypair generation, encryption, decryption) is what we all need and as per the need, the goal is to reduce *libcrypto.lib *to less than 3 MB. Using the generated .exe file is not an option. Please suggest ways to reduce the libcrypto.lib size to less than 3 MB on this 64 bit machine keeping only RSA functionality. And, is it possible by any chance that the size of libcrypto.lib will be smaller if OpenSSL is being built on a Windows 32 bit machine using a Windows 32 bit configuration option VC-WIN32? Thanks and Regards, Prithiraj -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
[openssl-users] RSA Public Key error
Hi, I have a RSA public key(PKCS 1v1.5) that I have obtained from somewhere. That key has been obtained after removing the first 24 bytes from the originally generated RSA public key. Those 24 bytes are being replaced by some custom 16 byte information which is being used as some sort of identifier in some future task and those 16 bytes are playing no role in encryption. OpenSSL fails to read this key. asn1parse shows some parsing error and most importantly RSA encryption in OpenSSL using this key fails. The untampered version of the RSA public key generated from the same source and containing the original 24 bytes at the beginning of the key is successfully read by OpenSSL and the RSA encryption using that key is also successful in OpenSSL. But our requirement is to use the first key containing the custom 16 byte information. My understanding is that the first 24 bytes of RSA public key following PKCS standards doesn't contain the modulus and exponent details required for RSA encryption. But OpenSSL seems to require these 24 bytes for encryption. Can someone please confirm what kind of information is present in the first 24 bytes of RSA Public key and/or why does OpenSSL need it? If possible, please suggest a solution to work with that RSA public key containing custom 16 byte information at the beginning of the key. Thanks and Regards, Prithiraj -- openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
