Chris, I will follow-up as soon as I have the answers.
Thanks, Edgar -----Original Message----- From: Christopher R. Hertel [mailto:c...@samba.org] Sent: Saturday, June 18, 2011 10:53 PM To: Edgar Olougouna Cc: cifs-proto...@samba.org Subject: Re: [cifs-protocol] [REG:111061756137964] Encryption of the key for "netsh branchcache” importkey and exportkey. Okay, I'm wrong. The IV does make a difference. So... My questions again: QUESTION: From what input is the Initialization Vector (IV) derived? Is it constant, derived from the initial passphrase, or derived from the SHA256 of the passphrase? QUESTION: What algorithm is used to derive the IV or, if the IV is constant, what values are used? QUESTION: Is the passphrase handled as Unicode or ASCII? I'm Assuming Unicode. QUESTION: Is the AES mode used AES-CBC? AES-ECB? Other? Two More QUESTION: There isn't any salt used, is there? QUESTION: You did write "prepended", so if the file I exported is 80 bytes long, it should be: [32-bytes SHA256(X)] + [48-bytes X] ...where X is the server secret. I just can't seem to get this to decipher, and it's probably just something basic that I'm missing... like the IV. Chris -)----- Christopher R. Hertel wrote: > Okay, never mind the IV part. I don't think that the IV actually > makes a difference when decrypting. > > ...but I could be wrong. > > Christopher R. Hertel wrote: >> Edgar, >> >> There is a little more information that I will need in order to make >> this work. In particular, the AES encryption algorithm makes use of >> an Initialization Vector which is typically derived from the >> passphrase, as is the key. >> >> Your description tells me how the key is derived, but not how the IV is >> derived. >> >> QUESTION: Is the IV derived from the initial passphrase or from the >> SHA256 of the passphrase? ...or is it a constant? >> >> QUESTION: What algorithm is used to derive the IV or, if the IV is >> constant (including NULL), what values are used? >> >> QUESTION: Is the passphrase handled as Unicode or ASCII? >> I'm Assuming Unicode. >> >> QUESTION: Is the AES mode used AES-CBC? AES-ECB? Other? >> >> Thanks. >> >> Chris -)----- >> >> Edgar Olougouna wrote: >>> Chris, >>> >>> The algorithm is as follows. >>> >>> Let’s assume, Bs is the server configured binary string (or whatever is >>> generated if none was configured) for the secret. >>> Let’s assume, Ks is the server secret, by definition Ks is SHA-256 hash of >>> Bs. >>> >>> When you run "netsh branchcache exportkey”, the key file is encrypted as >>> follows: >>> • A buffer is generated, consisting of Bs prepended with Ks. >>> • The buffer is encrypted using the AES-256 algorithm. The SHA-256 hash >>> of the passphrase string, excluding the string’s NULL terminator, is used >>> as the encryption key. >>> • The resulting encrypted buffer is written to the output file. >>> >>> During import (netsh branchcache importkey), the input file (which was the >>> output file in the export) is decrypted using the SHA-256 hash of the >>> passphrase as key. The value of Bs is validated against Ks, which is the >>> SHA-256 hash of Bs. >>> >>> NOTE: >>> I am pretty sure you have seen this is the document, but for a refresher >>> for a larger audience, the server secret key is defined in MS-PCCRC, as a >>> SHA-256 hash of an arbitrary length binary string (see excerpts below). >>> >>> MS-PCCRC >>> >>> 1.1 Glossary >>> server secret: A SHA-256 hash of an arbitrary length binary string stored >>> on the server. >>> 1.3 Overview >>> In order to ensure that cache content retrieval communications are at least >>> as secure as a normal client's communications with a content server, all >>> content servers must be configured with a binary secret value of arbitrary >>> length. This secret is used as a key to derive secret keys to be used to >>> secure communications between the peers or between peers and a Hosted >>> Cache. If the secret value is not configured, it is automatically generated. >>> >>> Best regards, >>> Edgar >>> >>> -----Original Message----- >>> From: Edgar Olougouna >>> Sent: Friday, June 17, 2011 10:49 AM >>> To: 'Christopher R. Hertel' >>> Cc: cifs-proto...@samba.org >>> Subject: RE: [cifs-protocol] [REG:111061069959469] BranchCache and SMB2: >>> Questions specific to BranchCache. >>> >>> Chris, >>> >>> Regarding the question on the server secret, I will be replying shortly and >>> describe the algorithm in a new thread [REG:111061756137964] Encryption of >>> the key for "netsh branchcache” importkey and exportkey. >>> That way you can achieve the decryption of the exported server secret and >>> verify your implementation. >>> >>> Regarding your feedback on the glossary definition of "segment secret" in >>> [MS-PCCRC]... I have passed on your observations to the product team. >>> >>> Thanks, >>> Edgar >>> >>> -----Original Message----- >>> From: Christopher R. Hertel [mailto:c...@ubiqx.com] >>> Sent: Friday, June 17, 2011 12:11 AM >>> To: Edgar Olougouna >>> Cc: cifs-proto...@samba.org >>> Subject: Re: [cifs-protocol] [REG:111061069959469] BranchCache and SMB2: >>> Questions specific to BranchCache. >>> >>> Edgar, >>> >>> Thanks for the clarifications. >>> >>> Regarding the dwOffsetInFirstSegment and dwReadBytesInLastSegment fields... >>> >>> I have a working prototype, written as a CGI script and running under >>> Apache. It seems to be giving correct results, when compared to a W2K8r2 >>> server, but I cannot verify that I am correctly generating the Segment >>> Secret correctly because I do not know the Server Secret generated by the >>> Windows server. >>> >>> I can extract the Windows Server Secret using a netsh command, but the >>> command encrypts the Server Secret with a password, and I do not know what >>> algorithm is used to perform the encryption. If I did, I could decrypt the >>> Server Secret and use it in my test program to verify that I am generating >>> the hash correctly. >>> >>> Do you know where I could find documentation that would tell me what >>> algorithm is being used to encrypt the extracted Server Secret? >>> >>> >>> >>> Regarding the glossary definition of "segment secret" in [MS-PCCRC]... >>> >>> I would suggest that the use of "authenticated" in the context of the >>> description of "segment secret" is misleading. [MS-PCCRC] covers a variety >>> of server types, and in many cases there is no authentication required. >>> I'm testing using HTTP, for example, and my client is being given Content >>> Information without performing any sort of authentication. >>> >>> The fact that SMB2 requires authentication before providing access to a >>> content or content information is an aspect of SMB2, not BranchCache. >>> There is no authorization required by the Peer Content Caching and >>> Retrieval: >>> Content Identification protocol itself. >>> >>> Thanks. >>> >>> See you next week. >>> >>> Chris -)----- >>> >>> >>> Edgar Olougouna wrote: >>>> Chris, >>>> >>>> For the question regarding "authorized client", BranchCache considers >>>> possession of the Content Information data structure to be equivalent to >>>> possession of data for access control purposes. If a client was able to >>>> obtain hashes from a file server, we will allow that client to retrieve >>>> data from peers or the hosted cache server. A file server is expected to >>>> protect the content information data structure the same way it protects >>>> actual files. If a file server authenticates and authorizes clients >>>> before allowing them to read files, it should do the same before providing >>>> them with hashes. >>>> >>>> Yes, your interpretation regarding dwOffsetInFirstSegment and >>>> dwReadBytesInLastSegment appears to be correct. >>>> Basically, dwReadBytesInLastSegment is a count of bytes you can read in >>>> the last segment. If the range is one segment long then the last segment >>>> is the only segment so dwOffsetInFirstSegment and dwReadBytesInLastSegment >>>> both apply to it. If the range is multiple segments long then >>>> dwOffsetInFirstSegment applies to the first segment and >>>> dwReadBytesInLastSegment applies to the last segment. >>>> >>>> In my previous communication, it should be Windows Server 2008 R2 instead >>>> of Windows 7. Only Windows Server 2008 R2 servers could be BranchCache >>>> servers, so both OS versions cannot be used interchangeably in this case. >>>> Thanks for catching this. >>>> >>>> Finally, I have passed on your feedback to the product team and suggested >>>> that an example of range request being added to the document. >>>> Thanks again for the numerous observations. >>>> >>>> Regards, >>>> Edgar >>>> >>>> -----Original Message----- >>>> From: Edgar Olougouna >>>> Sent: Friday, June 10, 2011 2:14 PM >>>> To: 'Christopher R. Hertel' >>>> Cc: cifs-proto...@samba.org >>>> Subject: RE: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: >>>> Questions specific to BranchCache. >>>> >>>> Chris, >>>> >>>> Thanks for the numerous observations. I will review the questions and >>>> follow-up as soon as I have an update. >>>> >>>> Regards, >>>> Edgar >>>> >>>> -----Original Message----- >>>> From: Christopher R. Hertel [mailto:c...@ubiqx.com] >>>> Sent: Friday, June 10, 2011 12:57 AM >>>> To: Edgar Olougouna >>>> Cc: cifs-proto...@samba.org >>>> Subject: Re: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: >>>> Questions specific to BranchCache. >>>> >>>> Edgar, >>>> >>>> One more observation and another quick question: >>>> >>>> Question: >>>> >>>> The [MS-PCCRC] Glossary (section 1.1) gives the following definition: >>>> >>>> segment secret: The content-specific hash that is sent to authorized >>>> clients along with the rest of the content information. It is >>>> generated by hashing the concatenation of the segment hash of data >>>> and the server secret. >>>> >>>> What does the phrase "authorized clients" mean? How is a client >>>> authorized in this context? >>>> >>>> Observation: >>>> Now that I understand the purpose of the dwOffsetInFirstSegment and >>>> dwReadBytesInLastSegment fields it appears that the descriptions of >>>> these fields in [MS-PCCRC], though inscrutable, are plausible. >>>> >>>> In particular, the following now makes sense: >>>> >>>> dwReadBytesInLastSegment (4 bytes): Total number of bytes of the >>>> content range which lie within the final segment in the Content >>>> Information data structure. >>>> >>>> If the requested range includes multiple segments, then the offset >>>> of the dwReadBytesInLastSegment bytes to be read is 0, relative to >>>> the segment. What is interesting to note is that if the requested >>>> range is included in only ONE segment, then the >>>> dwReadBytesInLastSegment bytes to be read start at the offset >>>> indicated by dwOffsetInFirstSegment. >>>> >>>> Chris -)----- >>>> >>>> >>>> Christopher R. Hertel wrote: >>>>> Edgar, >>>>> >>>>> Thanks again for this reply. I have some comments which I hope >>>>> you will pass along to the documentation team and engineers. >>>>> >>>>> On Tue, May 31, 2011 at 08:43:22PM +0000, Edgar Olougouna wrote: >>>>>> Chris, >>>>>> >>>>>> Please find our answers as follows. >>>>>> >>>>>> BranchCache divides content into 32 megabyte segments and further >>>>>> subdivides these segments into 64 kilobyte blocks. >>>>> Yes. [MS-PCCRC] does a good job of explaining that part. >>>>> >>>>>> The content information encodes: >>>>>> (1) ullOffsetInContent - the offset within the content at which >>>>>> the first segment present in the content information begins and >>>>> Again yes, this makes sense to me. >>>>> >>>>>> (2) dwOffsetInFirstSegment - the offset from ullOffsetInContent >>>>>> at which the first block of the first segment present in the >>>>>> content information begins. >>>>> This is where things get confusing. >>>>> >>>>>> Let's walk through this with an example. >>>>>> Let's assume the client requests a content range starting at >>>>>> content offset 33554433 (a.k.a. 32 megabytes plus one byte) and >>>>>> extending for >>>>>> 10 bytes. >>>>> Problem #1 (for me): The examples given in the document >>>>> specifically state that they show requests for entire files. From >>>>> section 3.1: >>>>> >>>>> "A client requests the entirety of the 125 kilobyte content from the >>>>> server. The server responds with Content Information of the following >>>>> form." >>>>> >>>>> >From section 3.2: >>>>> >>>>> "The same server S now receives a client request for a 125-megabyte >>>>> file." >>>>> >>>>> ...so, the examples in [MS-PCCRC] do not cover the use of ranges. >>>>> These are whole-file requests, so there is no example of how >>>>> dwOffsetInFirstSegment or dwReadBytesInLastSegment might be used >>>>> (though there are descriptions). >>>>> >>>>>> ullOffsetInContent will be 32 megabytes because that is where the >>>>>> first segment present in the content information begins. >>>>> Makes sense. >>>>> >>>>>> dwOffsetInFirstSegment will be 1 because the first segment in the >>>>>> content information is the second segment in the actual content >>>>>> because the requested content range starts at 32 megabytes plus one. >>>>> So... If I understand this correctly, the situation is that >>>>> BranchCache must operate in terms of Segments, and when it returns >>>>> the Segment Information for this segment, it is including an >>>>> offset, relative to the start of the Segment, at which the *requested* >>>>> content begins. >>>>> >>>>> Is that correct? >>>>> >>>>>> dwReadBytesInLastSegment will be 10 because that is the length of >>>>>> the content range that lies within the final segment present in >>>>>> the content information. >>>>> If my new understanding of dwOffsetInFirstSegment is correct, then >>>>> I believe that my understanding of dwReadBytesInLastSegment is now >>>>> also correctly updated. >>>>> >>>>> Basically, if the client requests a range the server must send >>>>> Content Information covering the segments that completely include >>>>> the requested range. These two values indicate how to handle the >>>>> cached segments when the client retrieves them from the cache, as follows: >>>>> >>>>> dwOffsetInFirstSegment - Discard this many leading bytes of the first >>>>> segment retrieved from cache. >>>>> >>>>> Retain all bytes of all subsequent segments >>>>> retrieved from cache except the last one. >>>>> >>>>> dwReadBytesInLastSegment - Retain only this many leading bytes of the >>>>> last segment retrieved from cache. >>>>> >>>>> Is that correct? >>>>> >>>>>> Note that dwReadBytesInLastSegment may be zero if the client did >>>>>> not request a content range (i.e. the client asked for the entire >>>>>> content). >>>>> So a dwReadBytesInLastSegment value of zero is interpreted as "all bytes" >>>>> in the segment. >>>>> >>>>>> The scenarios in which dwReadBytesInLastSegment will be zero are >>>>>> Windows specific behavior. In Windows 7, this happens whenever >>>>>> the content information covers the entire content. In order to >>>>>> participate in the MS-PCCRC protocol, a client must be able to >>>>>> parse content information that contains a zero value for >>>>>> dwReadBytesInLastSegment. The client must react by treating the >>>>>> segment size for the final segment encoded in the content >>>>>> information as the amount of data in the final segment. If >>>>>> dwReadBytesInLastSegment is non-zero then the client must only >>>>>> attempt to read dwReadBytesInLastSegment from the final segment, >>>>>> regardless of the segment size of the final segment. >>>>> Not quite... Assuming that I'm catching on here, and that such a >>>>> bug could happen, the goal would be something like the following: >>>>> >>>>> if 0 == dwReadBytesInLastSegment >>>>> readbytes = cbSegment >>>>> else >>>>> readbytes = min( dwReadBytesInLastSegment, cbSegment ) >>>>> >>>>> If, due to some bug, the dwReadBytesInLastSegment were greater >>>>> than the number of actual bytes in the segment then you would >>>>> still want to read only the number of actual bytes in the segment. >>>>> I can envision something like this happing if the requested range >>>>> extended beyond the end of the Content and the server did not clean up >>>>> the results properly. >>>>> >>>>> Okay, so I am being picky here... but it serves to verify whether >>>>> or not I am understanding these fields. >>>>> >>>>>> On Windows 7, if the content fits in one segment and the client >>>>>> issued a request for the entire content then, >>>>>> dwReadBytesInLastSegment should be zero and dwOffsetInFirstSegment >>>>>> should be zero. >>>>> It was my understanding, based on my reading of a number of >>>>> several MSDN web pages, that only Windows 2008R2 servers could be >>>>> BranchCache servers. >>>>> Windows 7 systems could only be clients. Vista and Windows 2008 >>>>> systems could also be clients if an additional package was installed. >>>>> (W2Kr2 servers can also be BranchCache clients, I believe.) >>>>> >>>>> The idea that Windows 7 cannot be a BranchCache Server is somewhat >>>>> supported by the following Windows Behavior note from [MS-PCCRC]: >>>>> >>>>> "<2> Section 2.3: In Windows 7, the [MS-PCCRR] implementation can only >>>>> accept segment IDs generated using SHA-256, whereas the Windows Server >>>>> 2008 R2 implementation of the hash generation part can generate segment >>>>> IDs using one of three hashing algorithms: SHA-256, SHA- 384, or >>>>> SHA-512." >>>>> >>>>> Note that the behavior note only talks about Windows 7 in client >>>>> mode and says nothing about its capabilities in server mode. I >>>>> believe that's because Windows 7 doesn't support server mode. >>>>> Also, only a Windows >>>>> 2008r2 server can act as a hosted cache. Windows 7 clients, I >>>>> believe, can operate as peer caches. >>>>> >>>>> So... Do you mean to say Windows 7 or Windows 2Kr2 in the statement >>>>> above? >>>>> If Windows 7, then are you talking about W7 clients sending >>>>> content requests to the hosted cache server or as broadcasts on the local >>>>> LAN? >>>>> If so, that part of BranchCache is outside of the scope of [MS-PCCRC] ... >>>>> which leaves me confused as to what you are trying to explain to >>>>> me when you describe Windows 7 behavior. >>>>> >>>>>> The product team will be reflecting the above clarification in >>>>>> the protocol document. Let me know whether this answers your questions. >>>>> It would be quite helpful if an additional example were presented >>>>> in order to show how these values are used. I found the >>>>> descriptions of these values to be difficult to interpret when reading >>>>> the documentation. >>>>> >>>>> Unless you tell me that my understanding, given above, is >>>>> incorrect I believe that your description has clarified things nicely. >>>>> >>>>> Thanks. >>>>> >>>>> Chris -)----- >>>>> >>>>>> Regards, >>>>>> Edgar >>>>>> >>>>>> -----Original Message----- >>>>>> From: Edgar Olougouna >>>>>> Sent: Tuesday, May 31, 2011 11:07 AM >>>>>> To: 'Christopher R. Hertel' >>>>>> Cc: cifs-proto...@samba.org >>>>>> Subject: RE: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: >>>>>> Questions specific to BranchCache. >>>>>> >>>>>> Chris, >>>>>> >>>>>> I am actively working on this with the product team to clarify a few >>>>>> more points. I will update you as soon as possible. >>>>>> >>>>>> Regards, >>>>>> Edgar >>>>>> >>>>>> -----Original Message----- >>>>>> From: Christopher R. Hertel [mailto:c...@ubiqx.com] >>>>>> Sent: Sunday, May 29, 2011 5:57 PM >>>>>> To: Edgar Olougouna >>>>>> Cc: MSSolve Case Email; cifs-proto...@samba.org >>>>>> Subject: Re: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: >>>>>> Questions specific to BranchCache. >>>>>> >>>>>> Edgar, >>>>>> >>>>>> I am just checking in on this ticket. I understand that the resolution >>>>>> to these issues can take some time, but I want to test my implementation >>>>>> at the FileSharing Plugfest in a few weeks time. Having the answers >>>>>> would help me complete my testing implementation. >>>>>> >>>>>> Thanks. >>>>>> >>>>>> Chris -)----- >>>>>> >>>>>> Obaid Farooqi wrote: >>>>>>> Hi Chris: >>>>>>> Thank you for contacting Microsoft regarding your query on branch >>>>>>> cache. A member of the protocol documentation team will be in touch >>>>>>> soon. >>>>>>> >>>>>>> Regards, >>>>>>> Obaid Farooqi >>>>>>> Escalation Engineer | Microsoft >>>>>>> >>>>>>> Exceeding your expectations is my highest priority. If you >>>>>>> would like to provide feedback on your case you may contact my >>>>>>> manager at allis...@microsoft.com >>>>>>> >>>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Christopher R. Hertel [mailto:c...@ubiqx.com] >>>>>>> Sent: Tuesday, May 17, 2011 9:12 PM >>>>>>> To: Interoperability Documentation Help >>>>>>> Subject: BranchCache and SMB2: Questions specific to BranchCache. >>>>>>> >>>>>>> I have a couple of questions regarding BranchCache and [MS-PCCRC]. As >>>>>>> you know, BranchCache--particularly the Content Information format >>>>>>> specified in [MS-PCCRC]--can be used as an extension to SMB2. >>>>>>> >>>>>>> 1) In the Content Information structure ([MS-PCCRC:2.3]), there is the >>>>>>> following field definition: >>>>>>> >>>>>>> dwOffsetInFirstSegment (4 bytes): Number of bytes into the first >>>>>>> segment within the Content Information data structure at which >>>>>>> the content range begins. >>>>>>> >>>>>>> I can't make heads nor tails of that description, and the captures I >>>>>>> have done on the wire typically show that field set to zero. >>>>>>> Similarly, the examples in section 3 of [MS-PCCRC] show the value of >>>>>>> this field to be zero. >>>>>>> >>>>>>> The field is 4 bytes in length, so it cannot be an offset into the >>>>>>> content itself, since content offsets are (should be) 8 bytes >>>>>>> (64-bit). >>>>>>> >>>>>>> Please clarify: What does dwOffsetInFirstSegment actually represent? >>>>>>> >>>>>>> 2) I am not getting what I expect in the dwReadBytesInLastSegment field. >>>>>>> This field is defined in [MS-PCCRC:2.3], as follows: >>>>>>> >>>>>>> dwReadBytesInLastSegment (4 bytes): Total number of bytes of the >>>>>>> content range which lie within the final segment in the Content >>>>>>> Information data structure. >>>>>>> >>>>>>> It seems to me that this value represents the number of bytes >>>>>>> of actual content included in the calculation of the final >>>>>>> Segment block of the Content Information. That interpretation >>>>>>> is supported by the examples in section 3. >>>>>>> >>>>>>> In actual captures, however, Windows IIS server returns zero >>>>>>> (0x00000000) in the dwReadBytesInLastSegment field. >>>>>>> >>>>>>> Is this a documentation bug or is the IIS server returning the >>>>>>> wrong value? >>>>>>> >>>>>>> This is IIS running on W2K8r2 with fairly up-to-date patches >>>>>>> applied. >>>>>>> >>>>>>> Looking forward to the replies. >>>>>>> >>>>>>> Thanks. >>>>>>> >>>>>>> Chris -)----- >>>>>>> >>>>>>> -- >>>>>>> "Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X >>>>>>> Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel >>>>>>> jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, >>>>>>> uninq. >>>>>>> ubiqx Team -- http://www.ubiqx.org/ -)----- c...@ubiqx.mn.org >>>>>>> OnLineBook -- http://ubiqx.org/cifs/ -)----- c...@ubiqx.org >>>>>>> >>>>>> -- >>>>>> Christopher R. Hertel -)----- Storage Architect & CIFS Geek >>>>>> http://www.ubiqx.com/ Data Storage and Systems Consulting >>>>>> "Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X >>>>>> > -- "Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq. ubiqx Team -- http://www.ubiqx.org/ -)----- c...@ubiqx.mn.org OnLineBook -- http://ubiqx.org/cifs/ -)----- c...@ubiqx.org _______________________________________________ cifs-protocol mailing list cifs-protocol@cifs.org https://lists.samba.org/mailman/listinfo/cifs-protocol
