On route to trying to fix the +heap warning, I actually fixed it first by using the attached patch.
I believe this patch is correct; it makes any error in the marshaling process sufficient to prevent an actual invocation. However, it's a fairly fundamental behavior change, and I don't know this code, so I thought I'd see if anyone objected before I submitted it. As far as I can tell, the behavior currently is that if we can't demarshal a particular argument in a typelib, we go ahead and call the function anyway. This could conceivably have 'good' side effects if the arguments we can't demarshall aren't used by the caller. The new behavior would be to fail anytime we don't understand a parameter in the typelib. That seems better to me, but again, I'm really new to this code, so kibitzing appreciated. Cheers, Jeremy
>From 7ba53d98859fb3fac7cc883e44f415e7982adc52 Mon Sep 17 00:00:00 2001 From: Jeremy White <[email protected]> Date: Sat, 24 Oct 2009 14:11:05 -0500 Subject: [PATCH] Report errors when [un]marshalling unknown types. --- dlls/oleaut32/tmarshal.c | 12 ++++++++---- 1 files changed, 8 insertions(+), 4 deletions(-) diff --git a/dlls/oleaut32/tmarshal.c b/dlls/oleaut32/tmarshal.c index e7d7b89..e81de94 100644 --- a/dlls/oleaut32/tmarshal.c +++ b/dlls/oleaut32/tmarshal.c @@ -909,7 +909,7 @@ serialize_param( } default: ERR("Unhandled marshal type %d.\n",tdesc->vt); - return S_OK; + return E_FAIL; } } @@ -1248,7 +1248,8 @@ deserialize_param( for (i=0;i<adesc->cDims;i++) arrsize *= adesc->rgbounds[i].cElements; for (i=0;i<arrsize;i++) - deserialize_param( + { + hres = deserialize_param( tinfo, readit, debugout, @@ -1257,6 +1258,9 @@ deserialize_param( (DWORD*)((LPBYTE)(arg)+i*_xsize(&adesc->tdescElem, tinfo)), buf ); + if (hres) + return hres; + } return S_OK; } case VT_SAFEARRAY: { @@ -1271,7 +1275,7 @@ deserialize_param( } default: ERR("No handler for VT type %d!\n",tdesc->vt); - return S_OK; + return E_FAIL; } } } @@ -2128,7 +2132,7 @@ TMStubImpl_Invoke( xargs += _argsize(&elem->tdesc, tinfo); if (hres) { ERR("Failed to deserialize param %s, hres %x\n",relaystr(names[i+1]),hres); - break; + goto exit; } } -- 1.5.6.5
