Re: B64 Decode issues (C)
Got some sample data to show which makes the bugger fail? On Thu, Aug 5, 2010 at 4:58 AM, Marc Phillips rm...@copacetic.net wrote: I mean if try to decode, it stops 15 bytes in. If I read the original file, and print out byte 15, it's a \n. I tried using BIO_FLAGS_BASE64_NO_NL, which then failes on encode. If I use it only on decode it just goes into an infinte loop of some kind. I'm thinking openssl might not be the most appropriate library for encoding/decoding files, as it functions fine for simple strings, but I'm having no luck with files (the file in question is around 5000 bytes but could easily be 4 times that size). R. Marc __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org -- Met vriendelijke groeten / Best regards, Ger Hobbelt -- web:http://www.hobbelt.com/ http://www.hebbut.net/ mail: g...@hobbelt.com mobile: +31-6-11 120 978 --
Re: B64 Decode issues (C)
Ger Hobbelt g...@hobbelt.com wrote: Got some sample data to show which makes the bugger fail? sqlite3 /tmp/MyTestsqlite.db SQLite version 3.3.6 Enter .help for instructions sqlite CREATE TABLE test (id INTEGER primary key AUTOINCREMENT, test TEXT); sqlite .quit encode: bmem = BIO_new(BIO_s_mem()); b64 = BIO_push(b64, bmem); fprintf(stderr,inbuf = %s, inlen = %d\n,inbuf,inlen); BIO_write(b64, inbuf, inlen); BIO_flush(b64); BIO_get_mem_ptr(b64, bptr); memcpy(outbuf, bptr-data, bptr-length); outbuf[bptr-length] = '\0'; BIO_free_all(b64); The code above yields the following encoded file: UAIGFysrAVl0YWJs ZXNxbGl0ZV9zZXF1ZW5jZXNxbGl0ZV9zZXF1ZW5jZQNDUkVBVEUgVEFCTEUgc3Fs aXRlX3NlcXVlbmNlKG5hbWUsc2VxKVgBBxcVFQGBE3RhYmxldGVzdHRlc3QCQ1JF QVRFIFRBQkxFIHRlc3QgKGlkIElOVEVHRVIgcHJpbWFyeSBrZXkgQVVUT0lOQ1JF TUVOVCwgdGVzdCBURVhUKQ0ABAAA AAANAAQA If I do the following I get back the db: cat /tmp/MyTestsqlite.db.b64 | openssl enc -base64 -d /tmp/MyTestsqlite.db.unb64 sqlite3 /tmp/MyTestsqlite.db.unb64 SQLite version 3.3.6 Enter .help for instructions sqlite .dump BEGIN
Re: B64 Decode issues (C)
Hi Marc, If you want to encrypt a database, have you considered SQLCipher (http://www.zetetic.net/code/sqlcipher)? It is free and uses OpenSSL. Jeff On Thu, Aug 5, 2010 at 11:19 AM, Marc Phillips rm...@copacetic.net wrote: Ger Hobbelt g...@hobbelt.com wrote: Got some sample data to show which makes the bugger fail? sqlite3 /tmp/MyTestsqlite.db SQLite version 3.3.6 Enter .help for instructions sqlite CREATE TABLE test (id INTEGER primary key AUTOINCREMENT, test TEXT); sqlite .quit encode: bmem = BIO_new(BIO_s_mem()); b64 = BIO_push(b64, bmem); fprintf(stderr,inbuf = %s, inlen = %d\n,inbuf,inlen); BIO_write(b64, inbuf, inlen); BIO_flush(b64); BIO_get_mem_ptr(b64, bptr); memcpy(outbuf, bptr-data, bptr-length); outbuf[bptr-length] = '\0'; BIO_free_all(b64); The code above yields the following encoded file: [SNIP] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: B64 Decode issues (C)
If you want to encrypt a database, have you considered SQLCipher (http://www.zetetic.net/code/sqlcipher)? It is free and uses OpenSSL. I'm not encrypting it (the data is encrypted via the app already anyway), I'm simply b64 encoding it; thanks though, this might be useful for other things. R. Marc __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: B64 Decode issues (C)
From: owner-openssl-us...@openssl.org On Behalf Of Marc Phillips Sent: Thursday, 05 August, 2010 11:20 Ger Hobbelt g...@hobbelt.com wrote: Got some sample data to show which makes the bugger fail? sqlite3 /tmp/MyTestsqlite.db SQLite version 3.3.6 Enter .help for instructions sqlite CREATE TABLE test (id INTEGER primary key AUTOINCREMENT, test TEXT); sqlite .quit encode: snipped The code above yields the following encoded file: snipped So you're encoding the whole db, not data in it. Linebreaks are good here, because this can be huge. If I do the following I get back the db: cat /tmp/MyTestsqlite.db.b64 | openssl enc -base64 -d /tmp/MyTestsqlite.db.unb64 (Aside: you don't need cat there. Just redirect 'enc' input.) sqlite3 /tmp/MyTestsqlite.db.unb64 snipped But if I use the code below to decode, It stops at byte 15: buffer = SQLite format 3, outlen = 15, inlen = 4160 Decode: b64 = BIO_new(BIO_f_base64()); bmem = BIO_new_mem_buf(inbuf, inlen); bmem = BIO_push(b64, bmem); outlen=BIO_read(bmem,outbuf,inlen); BIO_free_all(bmem); If I use exactly that code, with obvious declarations added, and inbuf/inlen = the encoded data you posted, I get outlen=3072 and outbuf contains data looking like what commandline produces (and you confirm is correct). You say 'buffer' = 'SQLite format 3', but there is no 'buffer' in your code. If you meant 'inbuf', that input is clearly not remotely similar to the encoded data you posted. Also it's suspicious to use inlen as max-read. While it's possible you've allocated your outbuf to be the same size as your encoded input (or greater) it's rather odd to do that. The outbuf size should be the amount of *decoded* data you expect and/or allow, and max-read should be that. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: B64 Decode issues (C)
From: owner-openssl-us...@openssl.org On Behalf Of Marc Phillips
Sent: Tuesday, 03 August, 2010 23:48
I'm having an issue with using base64 decode. I can encode fine,
and can decode strings just fine, but am having an issue
decoding (specifically) a sqlite database.
Here's my encode and decode: snip
As noted, encode and decode are fine on strings and some
binary files, but
if the string. Any decode seems to stop at the first \n.
I'm not sure quite what you mean here. bio_b64 by default
adds \n linebreaks every 64 chars when encoding (output)
and expects them at 80ish or less when decoding (input).
(IIRC the first PEM specs were 64 but some later specs
were higher, and this 'gap' allows OpenSSL to support all.)
Are you storing the string including the \n's, and getting it back
with them the same? Remember that if you run through C I/O,
\n gets changed to and from a possibly-different platform EOL;
in particular CRLF on MSDOS/Windows and CR on old(?) MacOS.
(And yet others on systems you are no longer likely to find.)
If you're writing/reading these strings on different systems --
perhaps because your database server is remote -- maybe some
translation happened.
I don't know what if anything sqlite does itself. If you are
encoding your OWN data to store and get back and decode later,
I would suggest not using ANY linebreaks. They sometimes confuse an
actual database, or database-related tools like report generators.
You can set BIO_FLAGS_BASE64_NO_NL, or you can (more simply) just
call EVP_{Encode,Decode}Block for memory-to-memory.
If I encode using the above routine, I can decode the file using
openssl enc -base64 -d
Yep, that expects (and -e produces) the 'standard' linebreaks.
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
Re: B64 Decode issues (C)
I'm not sure quite what you mean here. bio_b64 by default adds \n linebreaks every 64 chars when encoding (output) I mean if try to decode, it stops 15 bytes in. If I read the original file, and print out byte 15, it's a \n. I tried using BIO_FLAGS_BASE64_NO_NL, which then failes on encode. If I use it only on decode it just goes into an infinte loop of some kind. I'm thinking openssl might not be the most appropriate library for encoding/decoding files, as it functions fine for simple strings, but I'm having no luck with files (the file in question is around 5000 bytes but could easily be 4 times that size). R. Marc __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
