Re: [Mailman-Developers] GSoc - Requirement from Mentor to complete the project
Consider this flow of message from one user to all list-subscribers: * A user sends a message( signed as well as encrypted )which as soon as received by mailman is first queued in "IN" queue. * When the incoming runner wakes up the message is checked if it is for owners, posting or encrypted.An encrypted chain is created for encrypted messages. * Now according to the use case there can be two encrypted chain: 1) Where the posters are fully trusted and thus this rules only checks the signature of a message. If a message signed by a member is received it is sent to pipeline queue in a encrypted form for further processing. 2) Where the posters are not fully trusted. In this case there needs to be various checks on decrypted message like current default-posting-chain. * The messaged is moved in between queues in the encrypted form encrypted by a symmetric key algorithm which uses list's secret key to encrypt the message. * When the pipeline handler wakes up it decides which pipeline the message is to passed through. In this case an encrypted_posting_pipeline is created to process an encrypted message. * This encrypted_posting_pipeline is used to keep the message in plain-text for a minimal time. The message is decrypted, headers and footers are added and the message is signed by list's pub-key. It is then again (symmetrically) encrypted and stored in the outgoing queue. * The outgoing runner personalizes each message by encrypting it with each user's pub key and forwards it to MTA using VERP(Variable Envelop Return Path). Now consider the threat models it addresses. Firstly this verifies the integrity of the message. This is done by verification of signature and matching that sender's address indeed matches the address in signature so that you can sure that message was indeed sent by the sender. Secondly this maintains the confidentiality of the message. Even if the message is intercepted it can only be read by the one whom it is addressed to. There are the threats this does not address: If the keys/system/MUA of a user sending or receiving the message is compromised or contains a virus, no assurance can be provided about the integrity or confidentiality of the message. On Tue, May 7, 2013 at 8:27 AM, Stephen J. Turnbull wrote: > Abhilash Raj writes: > > > Let me divide the project in a few pieces so that each can be > > discussed upon separately. > > This is a good idea, but you should take them up one at a time, unless > you have a good approach and are expecting "sounds good, get started" > as the reply to that point. > > > * Firstly a utility to encrypt or decrypt the message. Well i found > > [python-gnupg][1] for this purpose and would try to write a wrapper > > around it for the use by mailman. But I found another option for it > > [GnupgInterface][2]. GnupgInterface was used in the > > [mailman-pgp-smime][3] patch for mailman and also has options to sign > > and encrypt in one call of a function( unlike python-gnupg ). If > > anyone has ever used any of these two would you please suggest which > > one is better? > > This isn't first. Don't be so eager to write code when you have not > stated the requirements with any precision. > > *First* you need to describe the life of a message from a thought in > the sender's mind until it hits the receiver's eyes. (It could > actually be somewhat shorter than that, but these endpoints ensure > you'll get everything we need somewhere in between.) Which steps are > required for every message? Which are optional, depending on the list > policy and/or user choices? Which are implemented in Mailman? Which > in MTAs/MDAs? Which in MUAs? > > You also need to decide what threats this process is suppose to > protect the users from. > > The combination of these two will determine what Mailman needs to be > able to do with incoming and outgoing posts. Then you need to see > what Mailman already does perfectly, what needs to be modified, and > what needs a new implementation. > > This description of requirements doesn't need to be as authoritative > as Scripture, but we need something fairly detailed to start with. > > Once that's done, we can talk about implementing crypto operations. > But I don't think it matters much which module you start with. (Why > not? The answer is a general concept of software engineering.) > > > * The point of encryption and decryption in the various queues. I was > > of the opinion that the message is decrypted as soon as it enters the > > IN queue and while its about to leave the queue it is encrypted with a > > symmetric key algorithm using the list's secret key. And then it is > > subsequently decrypted in the next queue and finally in the OUTGOING > > queue it is signed and encrypted with each user's pub-key. > > Any suggestions about this? > > I think the concerns about decrypted material hitting disk are valid. > > Therefore I would recommend that you avoid decrypting un
Re: [Mailman-Developers] GSoc - Requirement from Mentor to complete the project
Abhilash Raj writes: > Let me divide the project in a few pieces so that each can be > discussed upon separately. This is a good idea, but you should take them up one at a time, unless you have a good approach and are expecting "sounds good, get started" as the reply to that point. > * Firstly a utility to encrypt or decrypt the message. Well i found > [python-gnupg][1] for this purpose and would try to write a wrapper > around it for the use by mailman. But I found another option for it > [GnupgInterface][2]. GnupgInterface was used in the > [mailman-pgp-smime][3] patch for mailman and also has options to sign > and encrypt in one call of a function( unlike python-gnupg ). If > anyone has ever used any of these two would you please suggest which > one is better? This isn't first. Don't be so eager to write code when you have not stated the requirements with any precision. *First* you need to describe the life of a message from a thought in the sender's mind until it hits the receiver's eyes. (It could actually be somewhat shorter than that, but these endpoints ensure you'll get everything we need somewhere in between.) Which steps are required for every message? Which are optional, depending on the list policy and/or user choices? Which are implemented in Mailman? Which in MTAs/MDAs? Which in MUAs? You also need to decide what threats this process is suppose to protect the users from. The combination of these two will determine what Mailman needs to be able to do with incoming and outgoing posts. Then you need to see what Mailman already does perfectly, what needs to be modified, and what needs a new implementation. This description of requirements doesn't need to be as authoritative as Scripture, but we need something fairly detailed to start with. Once that's done, we can talk about implementing crypto operations. But I don't think it matters much which module you start with. (Why not? The answer is a general concept of software engineering.) > * The point of encryption and decryption in the various queues. I was > of the opinion that the message is decrypted as soon as it enters the > IN queue and while its about to leave the queue it is encrypted with a > symmetric key algorithm using the list's secret key. And then it is > subsequently decrypted in the next queue and finally in the OUTGOING > queue it is signed and encrypted with each user's pub-key. > Any suggestions about this? I think the concerns about decrypted material hitting disk are valid. Therefore I would recommend that you avoid decrypting until necessary. (It may not even be necessary. When? What exactly do I mean by "not decrypting"? This requires a good understand of the OpenPGP format, as well as Mailman queue processing. Don't hurry, feel free to ask questions -- as long as they don't amount to "I don't know, tell me"!) ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
Re: [Mailman-Developers] GSoc - Requirement from Mentor to complete the project
Let me divide the project in a few pieces so that each can be discussed upon separately. * Firstly a utility to encrypt or decrypt the message. Well i found [python-gnupg][1] for this purpose and would try to write a wrapper around it for the use by mailman. But I found another option for it [GnupgInterface][2]. GnupgInterface was used in the [mailman-pgp-smime][3] patch for mailman and also has options to sign and encrypt in one call of a function( unlike python-gnupg ). If anyone has ever used any of these two would you please suggest which one is better? Also from this [document][4] I see that there are two ways to both sign and encrypt the message: * It is stated that the data is first signed as multipart/signature body, and then encrypted to form the final multipart/encrypted body.[5] * A method for signing and encrypting data in a single OpenPGP message. This method is allowed in order to reduce processing overhead and increase compatibility with non-MIME implementations of OpenPGP. I don't have any links to prove, but I think we should use the first one. Any thoughts about which one is easily and more widely used among various MUAs? * Now the second part as wacky mentioned - a framework for storing the keys and handling the association of the key with a particular user. This same framework would be used to associate other methods of identification and authentication with the users. * The point of encryption and decryption in the various queues. I was of the opinion that the message is decrypted as soon as it enters the IN queue and while its about to leave the queue it is encrypted with a symmetric key algorithm using the list's secret key. And then it is subsequently decrypted in the next queue and finally in the OUTGOING queue it is signed and encrypted with each user's pub-key. Any suggestions about this? [1]: http://pythonhosted.org/python-gnupg/ [2]: http://search.cpan.org/~alexmv/GnuPG-Interface-0.46/lib/GnuPG/Interface.pm [3]: http://non-gnu.uvt.nl/mailman-pgp-smime/ [4]: https://tools.ietf.org/html/rfc3156#section-6 [5]: https://tools.ietf.org/html/rfc1847 On Sun, May 5, 2013 at 10:00 PM, Stephen J. Turnbull wrote: > Abhilash Raj writes: > > > 1) I need to discuss about the design of the idea that I want to > >implement. I know the rough bits but need help and comments on > >what I proposed. This would require views from the whole > >mailman-community. > > This is place for that, not private consultation with the mentor. > Just start spelling out your ideas, and post them here for comment as > they're developed to the point you can get useful comment. And don't > worry about whether you are doing it right: just do it. Sooner is better. > > One reason for this approach is that as Google sees GSoC, one purpose > is encouraging the student to develop his or her relationship with the > community.[1] The best case is somebody who enters with several months > of presence on the developer lists (but that is in no way required, > it's more an "it would be nice if" example). > > We actually would like have the proposals posted here, but > unfortunately that tends to interfere with evaluation for several > reasons. (One important one is that often several students propose > similar projects, and if the proposals are public there's a very > natural tendency for them to converge, making selection nearly > impossible.) Now that we have the proposals in full, we'd like you to > start engaging the community by posting your ideas, designs, and > questions here. > > It's probably best *not* to post big chunks of your proposal. Maybe > the "short description" as an introduction, but after that you should > pick a particular task and get the community to help you start on it. > > > 2) Apart from the corner cases mentioned in the proposal there might be > > many other not caught in my eye. I would need them to be pointed out so > > that a solution of it can discussed and then implemented. > > > > 3) There is a lot of cryptography in this project, so I would need a more > > experienced view to find and remove the security loopholes. > > These are more specific to your topic, but they are also best done on > this list, especially the issues related to security. The advantage > of the mentor system is that you have somebody who has to look at your > stuff. Still, the best way is to get comments from a broad selection > of developers and users. > > > 4) Lastly since I am still a noob in programming I would need regular > > review on the code for the first 2-3 weeks so that I cam improve > > and write bug free production code. > > Well, the first thing to learn is that nobody writes bug-free code. > Anticipate that your code is going to be buggy, catch as many as you > can systematically (I recommend Watts Humphrey's book "Introduction to > the Personal Software Process", I'm sure other mentors have their > favorites). Then get reviews from others (including bu
Re: [Mailman-Developers] GSoc - Requirement from Mentor to complete the project
Abhilash, Thanks for responding promptly and publicly. You, and the others should expect the discussions you mention without including them in response to my question. The discussion with your mentor is "given" -- That's one of the main responsibilities of a mentor. You may need to initiate community discussion about parts of your design to assure that your design meets the community expectations. But let me take your project as an example of the kind of input that I am seeking. You propose to use public key crypto in the processing stream of mail messages. In order to do so, you will need to associate the appropriate key with the corresponding user. You will also need to have some configuration parameters that describe the list policy and handling of the encryption. Now, I suggest that you need, but might reasonably expect someone else to provide, the following: 1) A framework for storing the keys and handling the association of the key with a particular user. This same framework would be used to associate other methods of identification and authentication with the users. You would be expected to implement the parts that are specific to public key. 2) A mechanism to display and edit parameters that describe the list policy as it refers to encryption. Again, as above, you would specify the parameters and provide the handlers that implement the policies, but you can expect someone else to handle the GUI presentation that turn them on and off, or otherwise configure them. I won't claim that the items that I suggest above are sufficient, or perhaps even necessary, but they can be used as examples for the kind of input that I am encouraging all of the applicants to discuss. There are other applicants who have proposed to develop, for example, GUI components. They should "step up" and declare that their proposed work will provide . Thus, we can observe that the projects complement each other. I hope to see this discussion continue. As for "mentor review" of your design. Let me point out that you do not have to be accepted by GSoC in order to contribute to MM development. I know that I, and I believe the other mentors, as well as other members of the community, would be quite willing to discuss "design" with anyone who wishes to help improve the overall system. That discussion can start at any time. We don't need to wait for the GSoC positions to be announced. Again, thanks for your participation. Richard On May 5, 2013, at 7:23 AM, Abhilash Raj wrote: > In response to Richard's mail these are the rough bits of what I will be > needing from my mentor if I am selected: > > 1) I need to discuss > Abhilash Raj ___ Mailman-Developers mailing list Mailman-Developers@python.org http://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9
