base64 decoding using an s_mem chain
Hi List-members,
The following problem has (in some form) popped up on this list repeatedly,
but after having browsed the archives until the beginning of this century I
didn't encounter (or simply missed?) a solution for my current problem, hence
the posting.
For some time now I'm trying to decode a base64 encoded file which is filtered
through a BIO_s_mem method. My intention is to write a function in which the
actual decoding is decoupled from the source of the encoded information as
well as from the destination of the decoded info, and so I thought of using a
BIO_s_mem method as an intermediate storage medium chained to a BIO_f_base64
method: obtain info the the source, put it into s_mem, base64 decode it, write
the decoded info to the destination.
As an initial attempt the following program does the trick, but a problem
occurs when I uncomment the section marked `Doesn't work'. In that case only
the first block of bytes that's read is decoded after which BIO_read
consistently returns 0.
The program as-is properly decodes base64 encoded information but requires me
to read all the encoded information into the s_mem buffer first, which is
unacceptable as it would require me to have all information available in
memory before base64 can start decoding.
So my questions are: What's the flaw in my reasoning (c.q. program)? And: what
must be done to decode information in a series of read-decode cycles rather
than using a `read-all, decode-all' procedure?
Here's the little program I used:
--
#include
#include
#include
int main()
{
BIO *bio, *b64;
char inbuf[500];
int inlen;
b64 = BIO_new(BIO_f_base64()); // define BIOs
BIO *mem = BIO_new(BIO_s_mem());
bio = BIO_push(b64, mem); // set up the chain
BIO_set_mem_eof_return(mem, 0); // define s_mem eof
// read info from some source
while ((inlen = fread(inbuf, 1, 500, stdin)) != 0)
{
BIO_write(mem, inbuf, inlen); // put it in the s_mem buffer
BIO_flush(mem);
// Doesn't work:
//while (1)
//{ // read what's already available
//inlen = BIO_read(bio, inbuf, inlen);
//if (inlen <= 0) // no more, then done
//break;
//// write decoded info to a dest.
//fwrite(inbuf, 1, inlen, stdout);
//}
}
// same procedure, but now write to the destination after first
// reading all info into s_mem
while (1)
{
inlen = BIO_read(bio, inbuf, 200);
if (inlen <= 0)
break;
fwrite(inbuf, 1, inlen, stdout);
}
BIO_free_all(bio);
}
--
Any suggestion I receive will of course greatly be appreciated.
Cheers,
--
Frank B. Brokken
Center for Information Technology, University of Groningen
(+31) 50 363 9281
Public PGP key: http://pgp.surfnet.nl
Key Fingerprint: 8E36 9FC4 1DAA FCDF 1A0D B19F DAC4 BE50 38C6 6170
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
Re: base64 decoding using an s_mem chain
On Tue December 30 2008, Frank B. Brokken wrote:
> Hi List-members,
>
> The following problem has (in some form) popped up on this list repeatedly,
> but after having browsed the archives until the beginning of this century I
> didn't encounter (or simply missed?) a solution for my current problem, hence
> the posting.
>
For implementing a solution using the openSSL libraries, I leave to the experts.
In general - consider reading any fixed length record -
Your input is sets of 4 octets, you can't have a split set -
Your output is sets of 3 octets, (a 4*6bit -> 3*8bit function) -
So your read function must either guarantee that a full records (4*x)
have been read or somehow handle the excess 1 - 3 octets as the
first part of the next read.
For a general purpose function, it is probably a poor idea to expect
the input to have line breaks (although some uses of base64 do have
line breaks - just not in general).
Same coding situation you would have if you where reading text lines,
any partial line read must be treated as the first part of the next
read operation.
Here is a page with a link to a public domain, base64 encode/decode
routine:
http://www.fourmilab.ch/webtools/base64/
(scroll down the page to the tar ball link)
Read it (I haven't) - see how they handled the situation.
Since those can be used in a pipeline, it must have an example of
the code you need in it.
Mike
> For some time now I'm trying to decode a base64 encoded file which is filtered
> through a BIO_s_mem method. My intention is to write a function in which the
> actual decoding is decoupled from the source of the encoded information as
> well as from the destination of the decoded info, and so I thought of using a
> BIO_s_mem method as an intermediate storage medium chained to a BIO_f_base64
> method: obtain info the the source, put it into s_mem, base64 decode it, write
> the decoded info to the destination.
>
> As an initial attempt the following program does the trick, but a problem
> occurs when I uncomment the section marked `Doesn't work'. In that case only
> the first block of bytes that's read is decoded after which BIO_read
> consistently returns 0.
>
> The program as-is properly decodes base64 encoded information but requires me
> to read all the encoded information into the s_mem buffer first, which is
> unacceptable as it would require me to have all information available in
> memory before base64 can start decoding.
>
> So my questions are: What's the flaw in my reasoning (c.q. program)? And: what
> must be done to decode information in a series of read-decode cycles rather
> than using a `read-all, decode-all' procedure?
>
> Here's the little program I used:
>
> --
> #include
> #include
> #include
>
> int main()
> {
> BIO *bio, *b64;
> char inbuf[500];
> int inlen;
>
> b64 = BIO_new(BIO_f_base64()); // define BIOs
> BIO *mem = BIO_new(BIO_s_mem());
>
> bio = BIO_push(b64, mem); // set up the chain
>
> BIO_set_mem_eof_return(mem, 0); // define s_mem eof
>
> // read info from some source
> while ((inlen = fread(inbuf, 1, 500, stdin)) != 0)
> {
> BIO_write(mem, inbuf, inlen); // put it in the s_mem buffer
> BIO_flush(mem);
>
> // Doesn't work:
> //while (1)
> //{ // read what's already available
> //inlen = BIO_read(bio, inbuf, inlen);
> //if (inlen <= 0) // no more, then done
> //break;
> //// write decoded info to a dest.
> //fwrite(inbuf, 1, inlen, stdout);
> //}
> }
>
> // same procedure, but now write to the destination after first
> // reading all info into s_mem
> while (1)
> {
> inlen = BIO_read(bio, inbuf, 200);
> if (inlen <= 0)
> break;
> fwrite(inbuf, 1, inlen, stdout);
> }
>
> BIO_free_all(bio);
> }
> --
>
>
> Any suggestion I receive will of course greatly be appreciated.
>
> Cheers,
>
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
Re: install openssl remote on linux
E, what's the web server software? Apache 2? Something else? In case of Apache v1 or v2, you very probably want to 'install' mod_ssl and then you need to install your server cert and pkey as well. Broadly speaking, that means you need access to the Apache config files at least, which is not webroot, at least not in my perception of what 'webroot' is (root dir of all web content (HTML files and such) and what is 'one above': is that '../' relatively speaking? Are all the httpd config files sitting there, or are they in /etc/httpd/ or in /... ? --> check distro manual for selected web server). Location of the Apache config files may differ slightly for each (Linux) distribution, and then, of course, the above only regards Apache 1 and 2 but there are many more web servers out there; each with their own setup/requirements, just like on Windows. ;-) Hence: best is to take this up with the linux server maintainer/admin and/or check vendor/distro manuals. Bottom line: most often, OpenSSL is already on there somewhere -- it's a library plus tools -- and you need to install the server extension/add-on/whatchammacallit which enables SSL functionality within the web server; then you need to configure the web server itself to actually offer SSL (HTTPS) connectivity, i.e. configure port 443 (open firewall too?), install the proper CA and server certificate files and server private key, and none of that should reside within the same space where you have your web content. Ger On Mon, Dec 29, 2008 at 2:55 PM, miek wrote: > Hi, > > > > I'm new to openssl and need to install it on a remote linux server. Of > course I don't have access to all server folders, only one above webroot. > > Firstly I'd like to know if this is possible, and second, how to install. > > I have downloaded the files from http://www.devside.net/guides/linux/openssl > but haven't found out how to install this remote. > > Is there an installer for linux like the one for windows? Or maybe you can > refer to a manual for situations like this…? > > > > Thanks for any help! > > regards, > > Miek > > -- Met vriendelijke groeten / Best regards, Ger Hobbelt -- web:http://www.hobbelt.com/ http://www.hebbut.net/ mail: g...@hobbelt.com mobile: +31-6-11 120 978 -- __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
vs2008 express + openssl 0.98i error when nmake
OS: windows vista ultimate visual studio 2008 express openssl 0.98i Error occurs when execute "nmake -f ms\ntdll.mak" I have add the VS bin directory into the PATH, the error message is : C:\openssl-0.9.8i>nmake -f ms\ntdll.mak Microsoft (R) Program Maintenance Utility Version 9.00.21022.08 Copyright (C) Microsoft Corporation. All rights reserved. Building OpenSSL cl /Fotmp32dll\uplink.obj -Iinc32 -Itmp32dll /MD /Ox /O2 /Ob2 /W3 /WX /G s0 /GF /Gy /nologo -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -DDS O_WIN32 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DOPENSSL_CPUID_O BJ -DOPENSSL_IA32_SSE2 -DAES_ASM -DBN_ASM -DOPENSSL_BN_ASM_PART_WORDS -DMD5_ASM -DSHA1_ASM -DRMD160_ASM -DOPENSSL_USE_APPLINK -I. /Fdout32dll -DOPENSSL_NO_CAMEL LIA -DOPENSSL_NO_SEED -DOPENSSL_NO_RC5 -DOPENSSL_NO_MDC2 -DOPENSSL_NO_TLSEXT -DO PENSSL_NO_CMS -DOPENSSL_NO_CAPIENG -DOPENSSL_NO_KRB5 -DOPENSSL_NO_DYNAMIC_ENGINE -D_WINDLL -c ms\uplink.c NMAKE : fatal error U1077: '"E:\Program Files\Microsoft Visual Studio 9.0\vc\bin \cl.EXE"' : return code '0xc135' Stop. HELP!
Re: Where to store client PEM certificates for an application
Victor Duchovni wrote: On Mon, Dec 29, 2008 at 12:55:14AM -0500, Edward Diener wrote: My assumptions from what I could glean from the certificates distributed is that the CA-cert.pem is the same for client and server, while the server-cert.pem is a public key corresponding to the private client-key.pem, and the server-key is a private key corresponding to the public client-cert. When I say "corresponding I mean that they form a public-private key pair. No it is simpler than that: For each (one or a few) server: - a server-cert.pem corresponds to a server-key.pem - in some deployments all servers share the same server cert and key - best practice is to generate the server keys on the server, and then obtain a CA cert for the public key (certificate request). For each (often many) clients: - a client-cert.pem corresponds to a client-key.pem - each and every client key and corresponding certificate pair are distinct from all other such pairs. - best practice is to generate the client keys on each client, and then obtain a CA cert for the public key (certificate request), you need to bootstrap secure (authenticated) delivery of the client CSR from the client to signing CA. The CA need not be a public CA, there is often little value in using a public CA in this context. - Client certificates are OPTIONAL. You can just encrypt the connection to the server and login with username/password. Makes sense. Thanks for the info. In fact from all the noise in this thread, it seems that simplicity is a major win whenever complexity is confusing, so dispense with the client certs entirely and go with username/password. TLS will just encrypt the login session and authenticate the server. In this last case I do not understand how the client can encrypt data going to the server if it has no private key of its own. If this is true then my client has a private key in its client-key.pem certificate. The premise is badly mangled, so it is difficult to comment on the conclusion. Your confusion is not OpenSSL confusion, it is basic lack of experience with public/private key security protocols and the roles the various keys play. Neither OpenSSL users, nor GnuTLS users, nor Microsoft CryptoAPI users, ... are specifically the right people to burden with your question. Are these certificates specifically SSL certificates or are they more broadly public key-private key certificates ? This is a general question about communication's security and requires independent research via books or a general security help forum. For what books do I look to specifically understand how these certificates work with public key-private key pairs ? SSL books ? Cryptography public key-private key books ? I am perfectly willing to learn, since I am a very fast learner, but I need to know where specifically to look. I do not want to have to learn all the details about computer security and cryptography, although that might be interesting in and of itself, but I do want to understand all the details about public key-private key and certificates. Unfortunately the MySQL documenation offers very little, which was the main reason for my OP on this NG. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Where to store client PEM certificates for an application
On Tue, Dec 30, 2008 at 02:26:20PM -0500, Edward Diener wrote: > Victor Duchovni wrote: > >On Mon, Dec 29, 2008 at 12:55:14AM -0500, Edward Diener wrote: > > > >>My assumptions from what I could glean from the certificates distributed > >>is that the CA-cert.pem is the same for client and server, while the > >>server-cert.pem is a public key corresponding to the private > >>client-key.pem, and the server-key is a private key corresponding to the > >>public client-cert. When I say "corresponding I mean that they form a > >>public-private key pair. > > > >No it is simpler than that: > > > >For each (one or a few) server: > > - a server-cert.pem corresponds to a server-key.pem > > - in some deployments all servers share the same server cert and key > > - best practice is to generate the server keys on the server, and > > then obtain a CA cert for the public key (certificate request). > > > >For each (often many) clients: > > - a client-cert.pem corresponds to a client-key.pem > > - each and every client key and corresponding certificate pair > > are distinct from all other such pairs. > > - best practice is to generate the client keys on each client, and > > then obtain a CA cert for the public key (certificate request), > > you need to bootstrap secure (authenticated) delivery of the client > > CSR from the client to signing CA. The CA need not be a public > > CA, there is often little value in using a public CA in this > > context. > > - Client certificates are OPTIONAL. You can just encrypt the > > connection to the server and login with username/password. > > Makes sense. Thanks for the info. > > > > >In fact from all the noise in this thread, it seems that simplicity is a > >major win whenever complexity is confusing, so dispense with the client > >certs entirely and go with username/password. TLS will just encrypt > >the login session and authenticate the server. > > In this last case I do not understand how the client can encrypt data > going to the server if it has no private key of its own. And yet it is possible, read up on Diffie-Hellman key-exchange. And learn more about SSL/TLS. You browse "https" websites with no private key of your own... > >Your confusion is not OpenSSL confusion, it is basic lack of experience > >with public/private key security protocols and the roles the various > >keys play. > > > >Neither OpenSSL users, nor GnuTLS users, nor Microsoft CryptoAPI users, ... > >are specifically the right people to burden with your question. > > Are these certificates specifically SSL certificates or are they more > broadly public key-private key certificates ? SSL != OpenSSL. And X.509 certificates are applicable more broadly than OpenSSL (e.g. IPSec and S/MIME). > >This is a general question about communication's security and requires > >independent research via books or a general security help forum. > > For what books do I look to specifically understand how these > certificates work with public key-private key pairs ? SSL books ? > Cryptography public key-private key books ? I am perfectly willing to > learn, since I am a very fast learner, but I need to know where > specifically to look. I do not want to have to learn all the details > about computer security and cryptography, although that might be > interesting in and of itself, but I do want to understand all the > details about public key-private key and certificates. > > Unfortunately the MySQL documenation offers very little, which was the > main reason for my OP on this NG. This mailing list is primarly for programmers developing applications that incorporate the OpenSSL APIs. Users of SSL-enabled products should generally seek help elsewhere, unless there is a specific issue with OpenSSL tools used in conjunction with said SSL-enabled products. The list is called "OpenSSL-users", not "SSL-users". -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: base64 decoding using an s_mem chain
Since base64 regroups the original 8-bits based binary into groups of 6 bits
for encoding, using padding as needed. So each original 8 bits is shared by two
6 bits, it is like a chain. To make your code work, you have to find out the
exact point, where a 8 bit is not shared. For example, change your buffer size
to "480", since 480 can be divided by both 8 and 6.
Good luck!
From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] On
Behalf Of Frank B. Brokken [f.b.brok...@rug.nl]
Sent: Tuesday, December 30, 2008 3:11 AM
To: openssl-users@openssl.org
Subject: base64 decoding using an s_mem chain
Hi List-members,
The following problem has (in some form) popped up on this list repeatedly,
but after having browsed the archives until the beginning of this century I
didn't encounter (or simply missed?) a solution for my current problem, hence
the posting.
For some time now I'm trying to decode a base64 encoded file which is filtered
through a BIO_s_mem method. My intention is to write a function in which the
actual decoding is decoupled from the source of the encoded information as
well as from the destination of the decoded info, and so I thought of using a
BIO_s_mem method as an intermediate storage medium chained to a BIO_f_base64
method: obtain info the the source, put it into s_mem, base64 decode it, write
the decoded info to the destination.
As an initial attempt the following program does the trick, but a problem
occurs when I uncomment the section marked `Doesn't work'. In that case only
the first block of bytes that's read is decoded after which BIO_read
consistently returns 0.
The program as-is properly decodes base64 encoded information but requires me
to read all the encoded information into the s_mem buffer first, which is
unacceptable as it would require me to have all information available in
memory before base64 can start decoding.
So my questions are: What's the flaw in my reasoning (c.q. program)? And: what
must be done to decode information in a series of read-decode cycles rather
than using a `read-all, decode-all' procedure?
Here's the little program I used:
--
#include
#include
#include
int main()
{
BIO *bio, *b64;
char inbuf[500];
int inlen;
b64 = BIO_new(BIO_f_base64()); // define BIOs
BIO *mem = BIO_new(BIO_s_mem());
bio = BIO_push(b64, mem); // set up the chain
BIO_set_mem_eof_return(mem, 0); // define s_mem eof
// read info from some source
while ((inlen = fread(inbuf, 1, 500, stdin)) != 0)
{
BIO_write(mem, inbuf, inlen); // put it in the s_mem buffer
BIO_flush(mem);
// Doesn't work:
//while (1)
//{ // read what's already available
//inlen = BIO_read(bio, inbuf, inlen);
//if (inlen <= 0) // no more, then done
//break;
//// write decoded info to a dest.
//fwrite(inbuf, 1, inlen, stdout);
//}
}
// same procedure, but now write to the destination after first
// reading all info into s_mem
while (1)
{
inlen = BIO_read(bio, inbuf, 200);
if (inlen <= 0)
break;
fwrite(inbuf, 1, inlen, stdout);
}
BIO_free_all(bio);
}
--
Any suggestion I receive will of course greatly be appreciated.
Cheers,
--
Frank B. Brokken
Center for Information Technology, University of Groningen
(+31) 50 363 9281
Public PGP key: http://pgp.surfnet.nl
Key Fingerprint: 8E36 9FC4 1DAA FCDF 1A0D B19F DAC4 BE50 38C6 6170
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
Interesting article
I know MD5 was broken ages ago but this article expands on the theme - make your own legitimate-looking root CA: http://www.crunchgear.com/2008/12/30/md5-collision-creates-rogue-certificate-authority/ -- Thomas Hruska Shining Light Productions Home of BMP2AVI, Nuclear Vision, ProtoNova, and Win32 OpenSSL. http://www.slproweb.com/ __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: vs2008 express + openssl 0.98i error when nmake
From: owner-openssl-us...@openssl.org [mailto:owner-openssl-us...@openssl.org] On Behalf Of gao yi Sent: Tuesday, December 30, 2008 5:14 AM To: openssl-users@openssl.org Subject: vs2008 express + openssl 0.98i error when nmake OS: windows vista ultimate visual studio 2008 express openssl 0.98i Error occurs when execute "nmake -f ms\ntdll.mak" I have add the VS bin directory into the PATH, the error message is : C:\openssl-0.9.8i>nmake -f ms\ntdll.mak Microsoft (R) Program Maintenance Utility Version 9.00.21022.08 Copyright (C) Microsoft Corporation. All rights reserved. Building OpenSSL cl /Fotmp32dll\uplink.obj -Iinc32 -Itmp32dll /MD /Ox /O2 /Ob2 /W3 /WX /G s0 /GF /Gy /nologo -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -DDS O_WIN32 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DOPENSSL_CPUID_O BJ -DOPENSSL_IA32_SSE2 -DAES_ASM -DBN_ASM -DOPENSSL_BN_ASM_PART_WORDS -DMD5_ASM -DSHA1_ASM -DRMD160_ASM -DOPENSSL_USE_APPLINK -I. /Fdout32dll -DOPENSSL_NO_CAMEL LIA -DOPENSSL_NO_SEED -DOPENSSL_NO_RC5 -DOPENSSL_NO_MDC2 -DOPENSSL_NO_TLSEXT -DO PENSSL_NO_CMS -DOPENSSL_NO_CAPIENG -DOPENSSL_NO_KRB5 -DOPENSSL_NO_DYNAMIC_ENGINE -D_WINDLL -c ms\uplink.c NMAKE : fatal error U1077: '"E:\Program Files\Microsoft Visual Studio 9.0\vc\bin \cl.EXE"' : return code '0xc135' Stop. HELP! [] Is your OS 64-bit (path to lib)? Did you set the Visual C variables with vcvarsall.bat? Two likely culprits.
Re: Interesting article
On Tue, Dec 30, 2008 at 06:38:24PM -0700, Thomas J. Hruska wrote: > I know MD5 was broken ages ago but this article expands on the theme - > make your own legitimate-looking root CA: To be precise, not a root CA, but an intermediate CA, from an issuing CA involved in multiple "unfortunate" practices. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
AES support in OPENSSL
Hello, Does OPENSSL support AES? Specificaly, via the evp API. The documentation does not indicate it is supported but I saw a few TLS functions witch use AES. Tx, Yaron.
