[openssl.org #3182] Bug in OpenSSL 1.0.1e 586 assembly optimized AES_cbc_encrypt

[Charles Osborne via RT]

I've noticed what appears to be a bug in the 586 assembly-optimized 
AES_cbc_encrypt function of OpenSSL 1.0.1e, (compiled on Windows) when 
encrypting data that is> 1 block in length, but not an integral multiple of the 
block size. Specifically it appears that when encrypting the partial-block 
"tail", the block is XOR-ed with the *original* IV passed to AES_cbc_encrypt, 
rather than the previous ciphertext block. This results in incorrect output 
when decrypting.

To test this, I encrypted 40 bytes (2 full blocks plus a half-block "tail") of 
zeros with a 128-bit all-zeros key (key-size does not appear to be a factor but 
provided for reproducability), and all-zeros initial IV. The output is as 
follows:

66 E9 4B D4 EF 8A 2C 3B 88 4C FA 59 CA 34 2B 2E
F7 95 BD 4A 52 E2 9E D7 13 D3 13 FA 20 E9 8D BC
66 E9 4B D4 EF 8A 2C 3B 88 4C FA 59 CA 34 2B 2E

Note that the last ciphertext block is identical to the first ciphertext block, 
which since the plaintext is the same (after the internal zero-padding that 
occurs before encrypting final partial-block) further indicates that it was 
encrypted using the same IV as the first block.

When decrypting this, the final block is corrupt:

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
F7 95 BD 4A 52 E2 9E D7 13 D3 13 FA 20 E9 8D BC

If instead the partial-block "tail" is encrypted separately to the full blocks, 
the ciphertext is:

66 E9 4B D4 EF 8A 2C 3B 88 4C FA 59 CA 34 2B 2E
F7 95 BD 4A 52 E2 9E D7 13 D3 13 FA 20 E9 8D BC
A1 0C F6 6D 0F DD F3 40 53 70 B4 BF 8D F5 BF B3

This decrypts to 3 blocks of zeros as expected.

Recompiling without assembly-optimized AES results in the expected 
functionality in both cases.

Reproduce code attached.

Thanks,

CO                                        

#include <stdio.h>
#include <string.h>
#include <openssl/aes.h>

void hexPrint(const void* pv, size_t len)
{
    const unsigned char * p = (const unsigned char*)pv;
    if (NULL == pv)
    {
        printf("NULL");
    }
    else
    {
        for (size_t i = 0; i < len; ++i, ++p)
        {
            if (i && (i % AES_BLOCK_SIZE) == 0) printf("\n");
            printf("%02x ", *p);
        }
    }
    printf("\n");
}

int main(int argc, char **argv)
{
    int keyLength = 128;
    
    unsigned char *aesKey = new unsigned char[keyLength / 8];
    memset(aesKey, 0, keyLength / 8);
    
    unsigned char ivEnc[AES_BLOCK_SIZE];
    unsigned char ivDec[AES_BLOCK_SIZE];

    memset(ivEnc, 0, AES_BLOCK_SIZE);
    memcpy(ivDec, ivEnc, AES_BLOCK_SIZE);
    
    // Will encrypt 2 full blocks plus an additional half block
    size_t inputLength = 2 * AES_BLOCK_SIZE + (AES_BLOCK_SIZE >> 1);
    size_t encLength = 3 * AES_BLOCK_SIZE;
    unsigned char *input = new unsigned char[inputLength];
    unsigned char *encOut = new unsigned char[encLength];
    unsigned char *decOut = new unsigned char[encLength];
    memset(input, 0, inputLength);
    memset(encOut, 0, encLength);
    memset(decOut, 0, encLength);

    AES_KEY encKey;
    AES_KEY decKey;
    AES_set_encrypt_key(aesKey, keyLength, &encKey);
    AES_cbc_encrypt(input, encOut, inputLength, &encKey, ivEnc, AES_ENCRYPT);

    printf("Encrypted:\n");
    hexPrint(encOut, encLength);

    AES_set_decrypt_key(aesKey, keyLength, &decKey);
    AES_cbc_encrypt(encOut, decOut, encLength, &decKey, ivDec, AES_DECRYPT);

    printf("Decrypted:\n");
    hexPrint(decOut, encLength);
    return 0;
}