Murray, Following on my last reply, I may also have replied a bit too quickly and forgot the last bit.
Your analysis about the context of the reply is correct. For calls like QueryRows, FindRow or ExpandRow, you need to maintain a state and track SetColumns requests call on the table object to be able to dump the reply properly. Given how the IDL of openchange is implemented, it is not something we can do at the NDR layer [1]. At mapiproxy level (after ndr decoding pass), given that we end up with a DATA_BLOB, it should be able to track the array of properties in the request, process the reply, then use the remaining bytes to create a new ndr pull context and pull remaining data to map the operations that got mangled by the NDR_REMAINING flag. What is the Outlook version where you have identified this behavior? If you have a reproducible test case, I would be very interested to know about it. [1] FYI, I have bootstrapped a while ago a subproject of openchange named mapirops which is a new IDL compiler that solves these problems. It is something at a very early stage, therefore probably not going to solve your issue in the short-term, but a promising lead in the mid-term: https://github.com/openchange/mapirops Julien. On 21/12/14 18:15, Murray McCulligh wrote: > Julien, > > I can take a look for sure. Not sure how much I can help with the > recipe this week, holidays will eliminate a lot of free time. > > I’m not currently only master (sitting on a code pull from a few > months ago). I’ll grab it and take a look. > > Murray > > *From:* Julien Kerihuel <mailto:[email protected]> > *Sent:* Sunday, December 21, 2014 12:05 PM > *To:* [email protected] <mailto:[email protected]> > *Subject:* Re: [openchange][devel] MAPIProxy Extraction To Structures > Issue > > Hi Murray, > > Actually I have just released a couple of tools in master recently > that let you exactly extract this information: > - rpcextract to extract rpc MAPI stub data into individual > request/response files > - mapipropsdump to do the context mapping and dump the DATA_BLOB > as a structure > > I have planned to release a technical recipe about this next week on > openchange.org, but maybe we can start the discussion on the list and > you can help writing the technical recipe? > > Julien. > > On 21/12/14 17:07, Murray McCulligh wrote: >> Hi all, >> >> I've come across an issue in MAPIProxy. It seems that if a response >> buffer contains a response (such as RopFindRow) that holds >> PropertyRow data, all responses after that in the buffer are not >> extracted into structures and are contained within the data blob. >> >> Unfortunately I don't see how MAPIProxy can easily fix this since I >> believe determining how much data the RopFindRow response contains >> requires decoding the properties and that might require knowing the >> requested order of properties set in a previous SetColumns request. >> >> Any ideas on how to address this would be welcome. >> >> Murray >> >> Dump of this occurring.... >> >> EcDoRpc: struct EcDoRpc >> in: struct EcDoRpc >> handle : * >> handle: struct policy_handle >> handle_type : 0x00000000 (0) >> uuid : >> 873babaf-1b2d-40eb-bcaa-33e92e80bf20 >> size : 0x00007fff (32767) >> offset : 0x00000000 (0) >> mapi_request : * >> mapi_len : 0x00000142 (322) >> length : 0x0122 (290) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x05 (5) >> logon_id : 0x01 (1) >> handle_idx : 0x00 (0) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 5) >> mapi_GetContentsTable: struct >> GetContentsTable_req >> handle_idx : 0x01 (1) >> TableFlags : 0x02 (2) >> 1: TableFlags_Associated >> 0: TableFlags_Depth >> 0: TableFlags_DeferredErrors >> 0: TableFlags_NoNotifications >> 0: TableFlags_SoftDeletes >> 0: TableFlags_UseUnicode >> 0: TableFlags_SuppressNotifications >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x12 (18) >> logon_id : 0x01 (1) >> handle_idx : 0x01 (1) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 18) >> mapi_SetColumns: struct SetColumns_req >> SetColumnsFlags : >> SetColumns_TBL_SYNC (0) >> prop_count : 0x0007 (7) >> properties: ARRAY(7) >> properties : >> PidTagFolderId (0x67480014) >> properties : PidTagMid >> (0x674A0014) >> properties : >> PidTagInstID (0x674D0014) >> properties : >> PidTagInstanceNum (0x674E0003) >> properties : >> PidTagMessageClass (0x1A001F) >> properties : >> PidTagConversationTopic (0x70001F) >> properties : >> PidTagReportName (0x3A001F) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x13 (19) >> logon_id : 0x01 (1) >> handle_idx : 0x01 (1) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 19) >> mapi_SortTable: struct SortTable_req >> SortTableFlags : 0x00 (0) >> lpSortCriteria: struct SSortOrderSet >> cSorts : 0x0001 (1) >> cCategories : 0x0000 (0) >> cExpanded : 0x0000 (0) >> aSort: ARRAY(1) >> aSort: struct SSortOrder >> ulPropTag : >> PidTagMessageClass (0x1A001F) >> ulOrder : >> TABLE_SORT_ASCEND (0x0) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x4f (79) >> logon_id : 0x01 (1) >> handle_idx : 0x01 (1) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 79) >> mapi_FindRow: struct FindRow_req >> ulFlags : DIR_FORWARD (0) >> res: struct mapi_SRestriction >> rt : 0x04 (4) >> res : union >> mapi_SRestriction_CTR(case 4) >> resProperty: struct >> mapi_SPropertyRestriction >> relop : 0x03 (3) >> ulPropTag : >> PidTagMessageClass (0x1A001F) >> lpProp: struct mapi_SPropValue >> ulPropTag : >> PidTagMessageClass (0x1A001F) >> value : >> union mapi_SPropValue_CTR(case 31) >> lpszW : >> 'IPM.Microsoft.PendingChange.' >> origin : >> BOOKMARK_BEGINNING (0) >> bookmark : SBinary_short cb=0 >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x02 (2) >> logon_id : 0x01 (1) >> handle_idx : 0x02 (2) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 2) >> mapi_OpenFolder: struct OpenFolder_req >> handle_idx : 0x03 (3) >> folder_id : >> 0x4c00000000000001 (5476377146882523137) >> OpenModeFlags : >> OpenModeFlags_Folder (0) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x29 (41) >> logon_id : 0x01 (1) >> handle_idx : 0x02 (2) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 41) >> mapi_RegisterNotification: struct >> RegisterNotification_req >> handle_idx : 0x04 (4) >> NotificationFlags : 0x0078 (120) >> 0: fnevCriticalError >> 0: fnevNewMail >> 0: fnevObjectCreated >> 1: fnevObjectDeleted >> 1: fnevObjectModified >> 1: fnevObjectMoved >> 1: fnevObjectCopied >> 0: fnevSearchComplete >> 0: fnevTableModified >> 0: fnevStatusObjectModified >> 0: fnevReserved >> 0: fnevTbit >> 0: fnevUbit >> 0: fnevSbit >> 0: fnevMbit >> WantWholeStore : 0x00 (0) >> FolderId : union >> hyperbool(case 0) >> ID : >> 0x4c00000000000001 (5476377146882523137) >> MessageId : union >> hyperbool(case 0) >> ID : >> 0x0000000000000000 (0) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x07 (7) >> logon_id : 0x01 (1) >> handle_idx : 0x03 (3) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 7) >> mapi_GetProps: struct GetProps_req >> PropertySizeLimit : 0x0000 (0) >> WantUnicode : 0x0001 (1) >> prop_count : 0x0012 (18) >> properties: ARRAY(18) >> properties : >> PidTagParentFolderId (0x67490014) >> properties : >> PidTagAccess (0xFF40003) >> properties : >> UNKNOWN_ENUM_VALUE (0x66720102) >> properties : >> PidTagDefaultPostMessageClass (0x36E5001F) >> properties : >> UNKNOWN_ENUM_VALUE (0x36E6001F) >> properties : >> PidTagDisplayName (0x3001001F) >> properties : >> PidTagFolderType (0x36010003) >> properties : >> PidTagContentCount (0x36020003) >> properties : >> PidTagContentUnreadCount (0x36030003) >> properties : >> PidTagSubfolders (0x360A000B) >> properties : >> PidTagContainerClass (0x3613001F) >> properties : >> UNKNOWN_ENUM_VALUE (0x36160102) >> properties : >> UNKNOWN_ENUM_VALUE (0x36DE0003) >> properties : >> UNKNOWN_ENUM_VALUE (0x36DF0102) >> properties : >> UNKNOWN_ENUM_VALUE (0x36E00102) >> properties : >> UNKNOWN_ENUM_VALUE (0x36E10003) >> properties : >> PidTagExtendedFolderFlags (0x36DA0102) >> properties : >> UNKNOWN_ENUM_VALUE (0x672D0003) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x01 (1) >> logon_id : 0x01 (1) >> handle_idx : 0x05 (5) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 1) >> mapi_Release: struct Release_req >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x03 (3) >> logon_id : 0x01 (1) >> handle_idx : 0x02 (2) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 3) >> mapi_OpenMessage: struct OpenMessage_req >> handle_idx : 0x06 (6) >> CodePageId : 0x0fff (4095) >> FolderId : >> 0x8814000000000001 (-8641281785017139199) >> OpenModeFlags : BestAccess (3) >> MessageId : >> 0x0f19000000000001 (1087900784986685441) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x2b (43) >> logon_id : 0x01 (1) >> handle_idx : 0x06 (6) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 43) >> mapi_OpenStream: struct OpenStream_req >> handle_idx : 0x07 (7) >> PropertyTag : >> PidTagRwRulesStream (0x68020102) >> OpenModeFlags : >> OpenStream_ReadOnly (0) >> mapi_request: struct EcDoRpc_MAPI_REQ >> opnum : 0x2c (44) >> logon_id : 0x01 (1) >> handle_idx : 0x07 (7) >> u : union >> EcDoRpc_MAPI_REQ_UNION(case 44) >> mapi_ReadStream: struct ReadStream_req >> ByteCount : 0x7c00 (31744) >> MaximumByteCount : union >> MaximumByteCount(case 31744) >> mapi_request : (handles) number=8 >> handle : 0x0000006a (106) >> handle : 0xffffffff >> (4294967295) >> handle : 0x00000047 (71) >> handle : 0xffffffff >> (4294967295) >> handle : 0xffffffff >> (4294967295) >> handle : 0x0000005b (91) >> handle : 0xffffffff >> (4294967295) >> handle : 0xffffffff >> (4294967295) >> length : * >> length : 0x0142 (322) >> max_data : 0x7fff (32767) >> EcDoRpc: struct EcDoRpc >> out: struct EcDoRpc >> handle : * >> handle: struct policy_handle >> handle_type : 0x00000000 (0) >> uuid : >> 873babaf-1b2d-40eb-bcaa-33e92e80bf20 >> size : 0x00007fff (32767) >> offset : 0x00000000 (0) >> mapi_response : * >> mapi_response : length=571 >> mapi_response: ARRAY(569) >> mapi_repl: struct EcDoRpc_MAPI_REPL >> opnum : 0x05 (5) >> handle_idx : 0x01 (1) >> error_code : MAPI_E_SUCCESS (0x0) >> u : union >> EcDoRpc_MAPI_REPL_UNION(case 5) >> mapi_GetContentsTable: struct >> GetContentsTable_repl >> RowCount : 0x0000000b (11) >> mapi_repl: struct EcDoRpc_MAPI_REPL >> opnum : 0x12 (18) >> handle_idx : 0x01 (1) >> error_code : MAPI_E_SUCCESS (0x0) >> u : union >> EcDoRpc_MAPI_REPL_UNION(case 18) >> mapi_SetColumns: struct SetColumns_repl >> TableStatus : >> TBLSTAT_COMPLETE (0) >> mapi_repl: struct EcDoRpc_MAPI_REPL >> opnum : 0x13 (19) >> handle_idx : 0x01 (1) >> error_code : MAPI_E_SUCCESS (0x0) >> u : union >> EcDoRpc_MAPI_REPL_UNION(case 19) >> mapi_SortTable: struct SortTable_repl >> TableStatus : >> TBLSTAT_COMPLETE (0) >> mapi_repl: struct EcDoRpc_MAPI_REPL >> opnum : 0x4f (79) >> handle_idx : 0x01 (1) >> error_code : MAPI_E_SUCCESS (0x0) >> u : union >> EcDoRpc_MAPI_REPL_UNION(case 79) >> mapi_FindRow: struct FindRow_repl >> RowNoLongerVisible : 0x00 (0) >> HasRowData : 0x01 (1) >> row : DATA_BLOB >> length=537 >> [0000] 01 00 01 00 00 00 00 00 14 88 00 01 00 00 00 00 ........ >> ........ >> [0010] 00 19 0E 00 01 00 00 00 00 00 19 0E 00 00 00 00 ........ >> ........ >> [0020] 00 00 49 00 50 00 4D 00 2E 00 52 00 75 00 6C 00 ..I.P.M. >> ..R.u.l. >> [0030] 65 00 2E 00 4D 00 65 00 73 00 73 00 61 00 67 00 e...M.e. >> s.s.a.g. >> [0040] 65 00 00 00 0A 0F 01 04 80 0A 0F 01 04 80 02 03 e....... >> ........ >> [0050] 00 00 00 00 00 00 29 04 00 00 00 00 07 03 00 00 ......). >> ........ >> [0060] 00 00 01 00 01 00 00 00 00 00 14 85 00 3F 00 00 ........ >> .....?.. >> [0070] 00 0A 0F 01 04 80 0A 0F 01 04 80 0A 0F 01 04 80 ........ >> ........ >> [0080] 00 43 00 61 00 6C 00 65 00 6E 00 64 00 61 00 72 .C.a.l.e >> .n.d.a.r >> [0090] 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 ........ >> ........ >> [00A0] 00 00 00 00 00 49 00 50 00 46 00 2E 00 41 00 70 .....I.P >> .F...A.p >> [00B0] 00 70 00 6F 00 69 00 6E 00 74 00 6D 00 65 00 6E .p.o.i.n >> .t.m.e.n >> [00C0] 00 74 00 00 00 0A 0F 01 04 80 00 00 00 00 40 0A .t...... >> ......@. >> [00D0] 0F 01 04 80 0A 0F 01 04 80 0A 0F 01 04 80 00 06 ........ >> ........ >> [00E0] 00 01 04 00 00 10 00 0A 0F 01 04 80 03 06 00 00 ........ >> ........ >> [00F0] 00 00 00 01 03 4F 75 74 6C 6F 6F 6B 20 52 75 6C .....Out >> look Rul >> [0100] 65 73 20 4F 72 67 61 6E 69 7A 65 72 00 00 00 00 es Organ >> izer.... >> [0110] 00 00 2B 07 00 00 00 00 F5 00 00 00 2C 07 00 00 ..+..... >> ....,... >> [0120] 00 00 F5 00 80 4F 12 00 80 4F 12 00 00 00 00 00 .....O.. >> .O...... >> [0130] 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 ........ >> ........ >> [0140] 00 00 00 00 01 00 00 00 01 00 00 00 00 00 00 00 ........ >> ........ >> [0150] 01 00 80 4F 12 05 26 43 00 6C 00 65 00 61 00 72 ...O..&C >> .l.e.a.r >> [0160] 00 20 00 63 00 61 00 74 00 65 00 67 00 6F 00 72 . .c.a.t >> .e.g.o.r >> [0170] 00 69 00 65 00 73 00 20 00 6F 00 6E 00 20 00 6D .i.e.s. >> .o.n. .m >> [0180] 00 61 00 69 00 6C 00 20 00 28 00 72 00 65 00 63 .a.i.l. >> .(.r.e.c >> [0190] 00 6F 00 6D 00 6D 00 65 00 6E 00 64 00 65 00 64 .o.m.m.e >> .n.d.e.d >> [01A0] 00 29 00 01 00 00 00 00 00 00 00 01 00 00 00 00 .)...... >> ........ >> [01B0] 00 00 00 01 00 00 00 4A 00 00 00 04 00 FF FF 00 .......J >> ........ >> [01C0] 00 0C 00 43 52 75 6C 65 45 6C 65 6D 65 6E 74 90 ...CRule >> Element. >> [01D0] 01 00 00 01 00 00 00 00 00 00 00 01 00 00 00 01 ........ >> ........ >> [01E0] 80 64 00 00 00 01 00 00 00 00 00 00 00 01 00 00 .d...... >> ........ >> [01F0] 00 01 80 F6 00 00 00 00 00 00 00 01 80 52 01 00 ........ >> .....R.. >> [0200] 00 00 00 00 00 00 00 00 00 00 00 00 00 66 66 66 ........ >> .....fff >> [0210] 66 E1 5D E4 40 00 00 00 00 f.].@... . >> mapi_response : (handles) number=8 >> handle id : 0x0000006a (106) >> handle id : 0x00000070 (112) >> handle id : 0x00000047 (71) >> handle id : 0x0000005a (90) >> handle id : 0x00000059 (89) >> handle id : 0x0000005b (91) >> handle id : 0x0000005b (91) >> handle id : 0x00000057 (87) >> length : * >> length : 0x025b (603) >> result : MAPI_E_SUCCESS (0x0) >> >> >> _______________________________________________ >> devel mailing list >> [email protected] >> http://mailman.openchange.org/listinfo/devel > > -- > Julien Kerihuel > [email protected] > OpenChange Project Founder > > Twitter: http://twitter.com/jkerihuel > > GPG Fingerprint: 0B55 783D A781 6329 108A B609 7EF6 FE11 A35F 1F79 > > ------------------------------------------------------------------------ > _______________________________________________ > devel mailing list > [email protected] > http://mailman.openchange.org/listinfo/devel > > > _______________________________________________ > devel mailing list > [email protected] > http://mailman.openchange.org/listinfo/devel -- Julien Kerihuel [email protected] OpenChange Project Founder Twitter: http://twitter.com/jkerihuel GPG Fingerprint: 0B55 783D A781 6329 108A B609 7EF6 FE11 A35F 1F79
signature.asc
Description: OpenPGP digital signature
_______________________________________________ devel mailing list [email protected] http://mailman.openchange.org/listinfo/devel
