Issue with newline

[Andrew Li]

Hi all,

I'm new to OpenSSL coding and am finding a behaviour that I cannot
understand.  Using the example code from the man page, when I try to
encrypt a single line, it works fine. But if I encrypt something like
"123\n\r12345\n\r", then I get a bad decrypt error:

bad decrypt
8788:error:06065064:digital envelope routines:EVP_DecryptFinal:bad
decrypt:evp_enc.c:445:

The file that the code generated looks correct, as in it's got the
correct size. Since block size is 8, and "123\n\r12345\n\r" is two
blocks long.

$ ls -l a
-rw-r--r--    1 68835  68835        16 Apr  1 08:52 a

I've attached the code that I used for encryption (no error checkings),
can someone tell me why it is behaving the way it is?

Thanks,
Andrew


#include <stdio.h>
#include <stdlib.h>
#include <openssl/evp.h>

int do_crypt(char *outfile, char *intext)
{
  unsigned char outbuf[1024];
  int outlen, tmplen;
  unsigned char key[] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
  unsigned char iv[] = {1,2,3,4,5,6,7,8};
  EVP_CIPHER_CTX ctx;
  FILE *out;
  EVP_CIPHER_CTX_init(&ctx);
  EVP_EncryptInit_ex(&ctx, EVP_bf_cbc(), NULL, key, iv);

  if(!EVP_EncryptUpdate(&ctx, outbuf, &outlen, intext, strlen(intext))) {
    /* Error */
    return 0;
  }
  if(!EVP_EncryptFinal_ex(&ctx, outbuf + outlen, &tmplen))
    {
      /* Error */
      return 0;
    }
  outlen += tmplen;
  EVP_CIPHER_CTX_cleanup(&ctx);

  out = fopen(outfile, "ab");
  fwrite(outbuf, 1, outlen, out);
  fclose(out);
  return 1;
}

int main(int argc, char *argv[]) {
  char *s;

  s = malloc(80);
  while(fgets(s, 80, stdin))
    do_crypt("/tmp/a", s);
  return 0;
}