[openssl.org #2519] crash when using newly-defined ASN1_SEQUENCE macros in Windows with elements that are also newly-defined SEQUENCEs...

[dave via RT]

configuration:
*  openssl 1.0.0.d
*  Win32, VC6
*  static library
 
(I don't know if the problem is confined to that configuration, but it's
what I'm using).
 
There appear to be two versions of opensslconf.h that are generated from
the Configure script.  One is placed in ./include/openssl, and one is
placed in ./crypto.  However, they are different; in particular with
respect to the definition of OPENSSL_EXPORT_VAR_AS_FUNCTION.  In the
'public' one placed in ./include/openssl, this is #undef'ed at line 79.
In the 'internal' one placed in ./crypto, this is #undef'ed but then
immediately #defined'ed at lines 84..85
 
The result is that, if the caller defines their own ASN1_SEQUENCE, and
uses that as a member of another ASN1_SEQUENCE, then the resulting code
for the _new() method will crash in tasn_new.c:51.
 
The reason is that the internal 'item' member of the template record
points to the template structure of the member sequence directly, rather
than to a function which returns the template of that member sequence.
 
Example of failing code:
 
//begin failing code=============
 
//defs==========
 
typedef struct {
 ASN1_OBJECT* type;
} INSIDE_SEQ;
 
ASN1_SEQUENCE(INSIDE_SEQ) = {
 ASN1_SIMPLE(INSIDE_SEQ, type, ASN1_OBJECT),
} ASN1_SEQUENCE_END(INSIDE_SEQ)
DECLARE_ASN1_FUNCTIONS(INSIDE_SEQ)
IMPLEMENT_ASN1_FUNCTIONS(INSIDE_SEQ)
 

typedef struct {
 ASN1_OBJECT* type;
 INSIDE_SEQ* member;
} OUTSIDE_SEQ;
 
ASN1_SEQUENCE(OUTSIDE_SEQ) = {
    ASN1_SIMPLE(OUTSIDE_SEQ, type, ASN1_OBJECT),
    ASN1_SIMPLE(OUTSIDE_SEQ, member, INSIDE_SEQ)
} ASN1_SEQUENCE_END(OUTSIDE_SEQ)
DECLARE_ASN1_FUNCTIONS(OUTSIDE_SEQ)
IMPLEMENT_ASN1_FUNCTIONS(OUTSIDE_SEQ)

 
//code==========
....
OUTSIDE_SEQ* pos = OUTSIDE_SEQ_new();    //crashes with access vioation
...
 
 
//end failing code====================
 
 
Complications:
*  since opensslconf.h #undef's the controlling constant, it isn't
possible to modify the behaviour with compiler defs, etc.
*  I didn't want to tamper with the generated opensslconf.h in case that
made things worse in an unexpected way.
 
I have made example code work by 'fixing up' the template record
definition manually, first by defining a function:
 
const ASN1_ITEM* INSIDE_SEQ_it_haquery(void)
{
return &INSIDE_SEQ_it;
} 
 
and then changing the 'const' data segment (where the template defs are
placed) to be writeable,
 
and then fixing up the template record this way:
 
 {
 ASN1_TEMPLATE* p = (ASN1_TEMPLATE*)&OUTSIDE_SEQ_seq_tt[1];
 p->item = (void*) INSIDE_SEQ_it_haquery;
 }

After dong that, then the call to:
 
OUTSIDE_SEQ* pos = OUTSIDE_SEQ_new();
 
will work as expected.  Of course, this is not really a good general
solution.
 
 
-dave

Title: Message

configuration:

*  openssl 1.0.0.d

*  Win32, VC6

*  static library

 

(I don't know if the problem is confined to that configuration, but it's what I'm using).

 

There appear to be two versions of opensslconf.h that are generated from the Configure script.  One is placed in ./include/openssl, and one is placed in ./crypto.  However, they are different; in particular with respect to the definition of OPENSSL_EXPORT_VAR_AS_FUNCTION.  In the 'public' one placed in ./include/openssl, this is #undef'ed at line 79.  In the 'internal' one placed in ./crypto, this is #undef'ed but then immediately #defined'ed at lines 84..85

 

The result is that, if the caller defines their own ASN1_SEQUENCE, and uses that as a member of another ASN1_SEQUENCE, then the resulting code for the _new() method will crash in tasn_new.c:51.

 

The reason is that the internal 'item' member of the template record points to the template structure of the member sequence directly, rather than to a function which returns the template of that member sequence.

 

Example of failing code:

 

//begin failing code=============

 

//defs==========

 

typedef struct {
 ASN1_OBJECT* type;
} INSIDE_SEQ;

 

ASN1_SEQUENCE(INSIDE_SEQ) = {
 ASN1_SIMPLE(INSIDE_SEQ, type, ASN1_OBJECT),
} ASN1_SEQUENCE_END(INSIDE_SEQ)
DECLARE_ASN1_FUNCTIONS(INSIDE_SEQ)
IMPLEMENT_ASN1_FUNCTIONS(INSIDE_SEQ)

 

typedef struct {
 ASN1_OBJECT* type;
 INSIDE_SEQ* member;
} OUTSIDE_SEQ;

 

ASN1_SEQUENCE(OUTSIDE_SEQ) = {
    ASN1_SIMPLE(OUTSIDE_SEQ, type, ASN1_OBJECT),
    ASN1_SIMPLE(OUTSIDE_SEQ, member, INSIDE_SEQ)
} ASN1_SEQUENCE_END(OUTSIDE_SEQ)
DECLARE_ASN1_FUNCTIONS(OUTSIDE_SEQ)
IMPLEMENT_ASN1_FUNCTIONS(OUTSIDE_SEQ)

 

//code==========

....

OUTSIDE_SEQ* pos = OUTSIDE_SEQ_new();    //crashes with access vioation

...

 

 

//end failing code====================

 

 

Complications:

*  since opensslconf.h #undef's the controlling constant, it isn't possible to modify the behaviour with compiler defs, etc.

*  I didn't want to tamper with the generated opensslconf.h in case that made things worse in an unexpected way.

 

I have made example code work by 'fixing up' the template record definition manually, first by defining a function:

 

const ASN1_ITEM* INSIDE_SEQ_it_haquery(void)
{
return &INSIDE_SEQ_it;
}

 

and then changing the 'const' data segment (where the template defs are placed) to be writeable,

 

and then fixing up the template record this way:

 

 {
 ASN1_TEMPLATE* p = (ASN1_TEMPLATE*)&OUTSIDE_SEQ_seq_tt[1];
 p->item = (void*) INSIDE_SEQ_it_haquery;
 }

After dong that, then the call to:

 

OUTSIDE_SEQ* pos = OUTSIDE_SEQ_new();

 

will work as expected.  Of course, this is not really a good general solution.

 

 

-dave