Julien, My module already does some tracking of SetColumns so this approach is possible for me. My module also splits request buffers so that I can send some requests, update my state from the responses, send injected requests if needed and then send the pending requests. So for now I’m just always splitting after I see a FindRow/QueryRow/GetProps request to avoid the issue.
I’m seeing this with Outlook 2007 and it is very reproducible. It usually occurs within the first flurry of requests after starting Outlook. I’ve been working on this module for a few months now and never noticed this until last week, when I noticed that I didn’t see responses for some OpenMessage requests. It is possible that my inbox now has the “correct” set of messages to create this situation. Won’t MAPIROPS have the same issue of needing to track state from things like SetColumns? Murray From: Julien Kerihuel Sent: Sunday, December 21, 2014 12:33 PM To: [email protected] Subject: Re: [openchange][devel] MAPIProxy Extraction To Structures Issue Murray, Following on my last reply, I may also have replied a bit too quickly and forgot the last bit. Your analysis about the context of the reply is correct. For calls like QueryRows, FindRow or ExpandRow, you need to maintain a state and track SetColumns requests call on the table object to be able to dump the reply properly. Given how the IDL of openchange is implemented, it is not something we can do at the NDR layer [1]. At mapiproxy level (after ndr decoding pass), given that we end up with a DATA_BLOB, it should be able to track the array of properties in the request, process the reply, then use the remaining bytes to create a new ndr pull context and pull remaining data to map the operations that got mangled by the NDR_REMAINING flag. What is the Outlook version where you have identified this behavior? If you have a reproducible test case, I would be very interested to know about it. [1] FYI, I have bootstrapped a while ago a subproject of openchange named mapirops which is a new IDL compiler that solves these problems. It is something at a very early stage, therefore probably not going to solve your issue in the short-term, but a promising lead in the mid-term: https://github.com/openchange/mapirops Julien. On 21/12/14 18:15, Murray McCulligh wrote: Julien, I can take a look for sure. Not sure how much I can help with the recipe this week, holidays will eliminate a lot of free time. I’m not currently only master (sitting on a code pull from a few months ago). I’ll grab it and take a look. Murray From: Julien Kerihuel Sent: Sunday, December 21, 2014 12:05 PM To: [email protected] Subject: Re: [openchange][devel] MAPIProxy Extraction To Structures Issue Hi Murray, Actually I have just released a couple of tools in master recently that let you exactly extract this information: - rpcextract to extract rpc MAPI stub data into individual request/response files - mapipropsdump to do the context mapping and dump the DATA_BLOB as a structure I have planned to release a technical recipe about this next week on openchange.org, but maybe we can start the discussion on the list and you can help writing the technical recipe? Julien. On 21/12/14 17:07, Murray McCulligh wrote: Hi all, I've come across an issue in MAPIProxy. It seems that if a response buffer contains a response (such as RopFindRow) that holds PropertyRow data, all responses after that in the buffer are not extracted into structures and are contained within the data blob. Unfortunately I don't see how MAPIProxy can easily fix this since I believe determining how much data the RopFindRow response contains requires decoding the properties and that might require knowing the requested order of properties set in a previous SetColumns request. Any ideas on how to address this would be welcome. Murray Dump of this occurring.... EcDoRpc: struct EcDoRpc in: struct EcDoRpc handle : * handle: struct policy_handle handle_type : 0x00000000 (0) uuid : 873babaf-1b2d-40eb-bcaa-33e92e80bf20 size : 0x00007fff (32767) offset : 0x00000000 (0) mapi_request : * mapi_len : 0x00000142 (322) length : 0x0122 (290) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x05 (5) logon_id : 0x01 (1) handle_idx : 0x00 (0) u : union EcDoRpc_MAPI_REQ_UNION(case 5) mapi_GetContentsTable: struct GetContentsTable_req handle_idx : 0x01 (1) TableFlags : 0x02 (2) 1: TableFlags_Associated 0: TableFlags_Depth 0: TableFlags_DeferredErrors 0: TableFlags_NoNotifications 0: TableFlags_SoftDeletes 0: TableFlags_UseUnicode 0: TableFlags_SuppressNotifications mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x12 (18) logon_id : 0x01 (1) handle_idx : 0x01 (1) u : union EcDoRpc_MAPI_REQ_UNION(case 18) mapi_SetColumns: struct SetColumns_req SetColumnsFlags : SetColumns_TBL_SYNC (0) prop_count : 0x0007 (7) properties: ARRAY(7) properties : PidTagFolderId (0x67480014) properties : PidTagMid (0x674A0014) properties : PidTagInstID (0x674D0014) properties : PidTagInstanceNum (0x674E0003) properties : PidTagMessageClass (0x1A001F) properties : PidTagConversationTopic (0x70001F) properties : PidTagReportName (0x3A001F) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x13 (19) logon_id : 0x01 (1) handle_idx : 0x01 (1) u : union EcDoRpc_MAPI_REQ_UNION(case 19) mapi_SortTable: struct SortTable_req SortTableFlags : 0x00 (0) lpSortCriteria: struct SSortOrderSet cSorts : 0x0001 (1) cCategories : 0x0000 (0) cExpanded : 0x0000 (0) aSort: ARRAY(1) aSort: struct SSortOrder ulPropTag : PidTagMessageClass (0x1A001F) ulOrder : TABLE_SORT_ASCEND (0x0) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x4f (79) logon_id : 0x01 (1) handle_idx : 0x01 (1) u : union EcDoRpc_MAPI_REQ_UNION(case 79) mapi_FindRow: struct FindRow_req ulFlags : DIR_FORWARD (0) res: struct mapi_SRestriction rt : 0x04 (4) res : union mapi_SRestriction_CTR(case 4) resProperty: struct mapi_SPropertyRestriction relop : 0x03 (3) ulPropTag : PidTagMessageClass (0x1A001F) lpProp: struct mapi_SPropValue ulPropTag : PidTagMessageClass (0x1A001F) value : union mapi_SPropValue_CTR(case 31) lpszW : 'IPM.Microsoft.PendingChange.' origin : BOOKMARK_BEGINNING (0) bookmark : SBinary_short cb=0 mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x02 (2) logon_id : 0x01 (1) handle_idx : 0x02 (2) u : union EcDoRpc_MAPI_REQ_UNION(case 2) mapi_OpenFolder: struct OpenFolder_req handle_idx : 0x03 (3) folder_id : 0x4c00000000000001 (5476377146882523137) OpenModeFlags : OpenModeFlags_Folder (0) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x29 (41) logon_id : 0x01 (1) handle_idx : 0x02 (2) u : union EcDoRpc_MAPI_REQ_UNION(case 41) mapi_RegisterNotification: struct RegisterNotification_req handle_idx : 0x04 (4) NotificationFlags : 0x0078 (120) 0: fnevCriticalError 0: fnevNewMail 0: fnevObjectCreated 1: fnevObjectDeleted 1: fnevObjectModified 1: fnevObjectMoved 1: fnevObjectCopied 0: fnevSearchComplete 0: fnevTableModified 0: fnevStatusObjectModified 0: fnevReserved 0: fnevTbit 0: fnevUbit 0: fnevSbit 0: fnevMbit WantWholeStore : 0x00 (0) FolderId : union hyperbool(case 0) ID : 0x4c00000000000001 (5476377146882523137) MessageId : union hyperbool(case 0) ID : 0x0000000000000000 (0) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x07 (7) logon_id : 0x01 (1) handle_idx : 0x03 (3) u : union EcDoRpc_MAPI_REQ_UNION(case 7) mapi_GetProps: struct GetProps_req PropertySizeLimit : 0x0000 (0) WantUnicode : 0x0001 (1) prop_count : 0x0012 (18) properties: ARRAY(18) properties : PidTagParentFolderId (0x67490014) properties : PidTagAccess (0xFF40003) properties : UNKNOWN_ENUM_VALUE (0x66720102) properties : PidTagDefaultPostMessageClass (0x36E5001F) properties : UNKNOWN_ENUM_VALUE (0x36E6001F) properties : PidTagDisplayName (0x3001001F) properties : PidTagFolderType (0x36010003) properties : PidTagContentCount (0x36020003) properties : PidTagContentUnreadCount (0x36030003) properties : PidTagSubfolders (0x360A000B) properties : PidTagContainerClass (0x3613001F) properties : UNKNOWN_ENUM_VALUE (0x36160102) properties : UNKNOWN_ENUM_VALUE (0x36DE0003) properties : UNKNOWN_ENUM_VALUE (0x36DF0102) properties : UNKNOWN_ENUM_VALUE (0x36E00102) properties : UNKNOWN_ENUM_VALUE (0x36E10003) properties : PidTagExtendedFolderFlags (0x36DA0102) properties : UNKNOWN_ENUM_VALUE (0x672D0003) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x01 (1) logon_id : 0x01 (1) handle_idx : 0x05 (5) u : union EcDoRpc_MAPI_REQ_UNION(case 1) mapi_Release: struct Release_req mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x03 (3) logon_id : 0x01 (1) handle_idx : 0x02 (2) u : union EcDoRpc_MAPI_REQ_UNION(case 3) mapi_OpenMessage: struct OpenMessage_req handle_idx : 0x06 (6) CodePageId : 0x0fff (4095) FolderId : 0x8814000000000001 (-8641281785017139199) OpenModeFlags : BestAccess (3) MessageId : 0x0f19000000000001 (1087900784986685441) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x2b (43) logon_id : 0x01 (1) handle_idx : 0x06 (6) u : union EcDoRpc_MAPI_REQ_UNION(case 43) mapi_OpenStream: struct OpenStream_req handle_idx : 0x07 (7) PropertyTag : PidTagRwRulesStream (0x68020102) OpenModeFlags : OpenStream_ReadOnly (0) mapi_request: struct EcDoRpc_MAPI_REQ opnum : 0x2c (44) logon_id : 0x01 (1) handle_idx : 0x07 (7) u : union EcDoRpc_MAPI_REQ_UNION(case 44) mapi_ReadStream: struct ReadStream_req ByteCount : 0x7c00 (31744) MaximumByteCount : union MaximumByteCount(case 31744) mapi_request : (handles) number=8 handle : 0x0000006a (106) handle : 0xffffffff (4294967295) handle : 0x00000047 (71) handle : 0xffffffff (4294967295) handle : 0xffffffff (4294967295) handle : 0x0000005b (91) handle : 0xffffffff (4294967295) handle : 0xffffffff (4294967295) length : * length : 0x0142 (322) max_data : 0x7fff (32767) EcDoRpc: struct EcDoRpc out: struct EcDoRpc handle : * handle: struct policy_handle handle_type : 0x00000000 (0) uuid : 873babaf-1b2d-40eb-bcaa-33e92e80bf20 size : 0x00007fff (32767) offset : 0x00000000 (0) mapi_response : * mapi_response : length=571 mapi_response: ARRAY(569) mapi_repl: struct EcDoRpc_MAPI_REPL opnum : 0x05 (5) handle_idx : 0x01 (1) error_code : MAPI_E_SUCCESS (0x0) u : union EcDoRpc_MAPI_REPL_UNION(case 5) mapi_GetContentsTable: struct GetContentsTable_repl RowCount : 0x0000000b (11) mapi_repl: struct EcDoRpc_MAPI_REPL opnum : 0x12 (18) handle_idx : 0x01 (1) error_code : MAPI_E_SUCCESS (0x0) u : union EcDoRpc_MAPI_REPL_UNION(case 18) mapi_SetColumns: struct SetColumns_repl TableStatus : TBLSTAT_COMPLETE (0) mapi_repl: struct EcDoRpc_MAPI_REPL opnum : 0x13 (19) handle_idx : 0x01 (1) error_code : MAPI_E_SUCCESS (0x0) u : union EcDoRpc_MAPI_REPL_UNION(case 19) mapi_SortTable: struct SortTable_repl TableStatus : TBLSTAT_COMPLETE (0) mapi_repl: struct EcDoRpc_MAPI_REPL opnum : 0x4f (79) handle_idx : 0x01 (1) error_code : MAPI_E_SUCCESS (0x0) u : union EcDoRpc_MAPI_REPL_UNION(case 79) mapi_FindRow: struct FindRow_repl RowNoLongerVisible : 0x00 (0) HasRowData : 0x01 (1) row : DATA_BLOB length=537 [0000] 01 00 01 00 00 00 00 00 14 88 00 01 00 00 00 00 ........ ........ [0010] 00 19 0E 00 01 00 00 00 00 00 19 0E 00 00 00 00 ........ ........ [0020] 00 00 49 00 50 00 4D 00 2E 00 52 00 75 00 6C 00 ..I.P.M. ..R.u.l. [0030] 65 00 2E 00 4D 00 65 00 73 00 73 00 61 00 67 00 e...M.e. s.s.a.g. [0040] 65 00 00 00 0A 0F 01 04 80 0A 0F 01 04 80 02 03 e....... ........ [0050] 00 00 00 00 00 00 29 04 00 00 00 00 07 03 00 00 ......). ........ [0060] 00 00 01 00 01 00 00 00 00 00 14 85 00 3F 00 00 ........ .....?.. [0070] 00 0A 0F 01 04 80 0A 0F 01 04 80 0A 0F 01 04 80 ........ ........ [0080] 00 43 00 61 00 6C 00 65 00 6E 00 64 00 61 00 72 .C.a.l.e .n.d.a.r [0090] 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 ........ ........ [00A0] 00 00 00 00 00 49 00 50 00 46 00 2E 00 41 00 70 .....I.P .F...A.p [00B0] 00 70 00 6F 00 69 00 6E 00 74 00 6D 00 65 00 6E .p.o.i.n .t.m.e.n [00C0] 00 74 00 00 00 0A 0F 01 04 80 00 00 00 00 40 0A .t...... ......@. [00D0] 0F 01 04 80 0A 0F 01 04 80 0A 0F 01 04 80 00 06 ........ ........ [00E0] 00 01 04 00 00 10 00 0A 0F 01 04 80 03 06 00 00 ........ ........ [00F0] 00 00 00 01 03 4F 75 74 6C 6F 6F 6B 20 52 75 6C .....Out look Rul [0100] 65 73 20 4F 72 67 61 6E 69 7A 65 72 00 00 00 00 es Organ izer.... [0110] 00 00 2B 07 00 00 00 00 F5 00 00 00 2C 07 00 00 ..+..... ....,... [0120] 00 00 F5 00 80 4F 12 00 80 4F 12 00 00 00 00 00 .....O.. .O...... [0130] 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 ........ ........ [0140] 00 00 00 00 01 00 00 00 01 00 00 00 00 00 00 00 ........ ........ [0150] 01 00 80 4F 12 05 26 43 00 6C 00 65 00 61 00 72 ...O..&C .l.e.a.r [0160] 00 20 00 63 00 61 00 74 00 65 00 67 00 6F 00 72 . .c.a.t .e.g.o.r [0170] 00 69 00 65 00 73 00 20 00 6F 00 6E 00 20 00 6D .i.e.s. .o.n. .m [0180] 00 61 00 69 00 6C 00 20 00 28 00 72 00 65 00 63 .a.i.l. .(.r.e.c [0190] 00 6F 00 6D 00 6D 00 65 00 6E 00 64 00 65 00 64 .o.m.m.e .n.d.e.d [01A0] 00 29 00 01 00 00 00 00 00 00 00 01 00 00 00 00 .)...... ........ [01B0] 00 00 00 01 00 00 00 4A 00 00 00 04 00 FF FF 00 .......J ........ [01C0] 00 0C 00 43 52 75 6C 65 45 6C 65 6D 65 6E 74 90 ...CRule Element. [01D0] 01 00 00 01 00 00 00 00 00 00 00 01 00 00 00 01 ........ ........ [01E0] 80 64 00 00 00 01 00 00 00 00 00 00 00 01 00 00 .d...... ........ [01F0] 00 01 80 F6 00 00 00 00 00 00 00 01 80 52 01 00 ........ .....R.. [0200] 00 00 00 00 00 00 00 00 00 00 00 00 00 66 66 66 ........ .....fff [0210] 66 E1 5D E4 40 00 00 00 00 f.].@... . mapi_response : (handles) number=8 handle id : 0x0000006a (106) handle id : 0x00000070 (112) handle id : 0x00000047 (71) handle id : 0x0000005a (90) handle id : 0x00000059 (89) handle id : 0x0000005b (91) handle id : 0x0000005b (91) handle id : 0x00000057 (87) length : * length : 0x025b (603) result : MAPI_E_SUCCESS (0x0) _______________________________________________ devel mailing list [email protected] http://mailman.openchange.org/listinfo/devel -- Julien Kerihuel [email protected] OpenChange Project Founder Twitter: http://twitter.com/jkerihuel GPG Fingerprint: 0B55 783D A781 6329 108A B609 7EF6 FE11 A35F 1F79 ------------------------------------------------------------------------------ _______________________________________________ devel mailing list [email protected] http://mailman.openchange.org/listinfo/devel _______________________________________________ devel mailing list [email protected] http://mailman.openchange.org/listinfo/devel -- Julien Kerihuel [email protected] OpenChange Project Founder Twitter: http://twitter.com/jkerihuel GPG Fingerprint: 0B55 783D A781 6329 108A B609 7EF6 FE11 A35F 1F79 -------------------------------------------------------------------------------- _______________________________________________ devel mailing list [email protected] http://mailman.openchange.org/listinfo/devel
_______________________________________________ devel mailing list [email protected] http://mailman.openchange.org/listinfo/devel
