Re: [Mailman-Developers] GSoC project - Implement module to process ARC headers

[Aditya Divekar]

>
>
> I/we need to find out exactly what we need.  We may want to do this
> through email-test.python.org (no-existent, for example, python.org is
> the relevant part, or maybe a private domain of someone's), but we
> have to be careful that it doesn't conflict with other uses.


email-test.python.org sounds apt. And since you think its good we could
ping Mark about the same.
Also, since its for our interop, we will be freeing it up again after
the testing period ( something along the line of a few months )
making it available again.


> I think that would need to be shortened, but mailauth would work and
> has a good rhythm. :-)  Of course it may be "taken".
>
>
Perfect! I checked PyPI and didn't find any such packages. So at least
till now no one has laid claim to it on PyPI though "taken" might mean
other things
I understand.
(Still, the doubt remains whether the original author would like to get the
name changed to something different. )


> Yes, but Murray's OpenDKIM is a C library, and I expect OpenArc is
> too.  I don't think we should be using a C library because we're not...
>


> In any case, the more independent implementations the better.
>

Yes, right. I somehow missed that C fact.
I'll like to see the code once its out, and see the approach used.
Not for our purposes, but just curious :)

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSoC project - Implement module to process ARC headers

[Aditya Divekar]

On Sat, Jun 18, 2016 at 4:08 PM, Stephen J. Turnbull 
wrote:

> Ah, that's a problem.  I don't have control of the DNS yet.  We'd need
> to interact with Mark.


So should a new conversation be started with Mark for
our request?


> I'll get you on the
> arc-interop list, there should be another one in the summer at the
> pace they've been going.
>

Okay!



> Not really.  Keywords that might be useful aside from ARC, DKIM, and
> DMARC: mail, post, verify, track, key, sign, identity.
>
> Ah -- one brainstorm -- kimpy = "key identified mail.py".
>
>
That sounds good! The `mail` keyword gave an idea - `emailauthentication`

As far as words go, you could just keep dkimpy, too, since ARC depends
> on domain keys for its semantics, as well as on DKIM for its signing
> protocol.  "adkimpy"?  Lots of double meanings there: Aditya, ARC,
> augmented, domain, divikar. :-)
>
>
Hahaha! In case we choose that, that can be our inside joke.

Narrowing it down, we have the options -
dkimpy, kimpy, adkimpy ( meaning augmented/arc - domain ;) ),
emailauthentication, and any others with keywords you specified.


> I was surprised by the news, but Google also chose dkimpy for its
> implementation of ARC.  I haven't had time to look at it, but maybe
> early next week (not publicly licensed, sorry, no URL).
>

I had read on the arc list that an OpenARC implementation was being
developed
by Murray Kucherawy.
So I suppose that we may have the news of it being released on the list
soon.

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSoC project - Implement module to process ARC headers

[Aditya Divekar]

>
>
> I don't understand what you're worried about.  Of course if you change
> all the names, especially with some sort of automatic process, there's
> a pretty good chance of introducing typos.  We fix them as we find
> them, that's all.
>

Yes, I went ahead and made the changes. I have a separate module
'arcpy' installed in my python dist-packages now which is basically the
refactored 'dkimpy' package. I don't think its the best of names since it
doesn't
give the idea of presence of DKIM signing/verifying features.
Any ideas for the same?

I have a server with a fixed IP which could just pipe mail to a Python
> script that reads it and passes the message to the ARC module for
> processing.  Then the script can send it back out if necessary.
>

That should work for us!
Also we would need to have a DMARC and DKIM record set up though for the
signing domain. The procedures for that are easy, and are outlined in
detail on the net.
So if we have a domain name associated with the ip, I'll go ahead and make
the TXT
records. Else, we'll have to create a new one.
I'm not aware of any way to avoid this. Let me know if you are aware of any
:)

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] GSoC project - Implement module to process ARC headers

[Aditya Divekar]

Hi!

It's been sometime since the GSoC coding period started, and I apologize
for putting in this mail so late!

The GSoC project is divided into two repositories.
ARC implementation - https://gitlab.com/adityadivekar/arc

refactored dkimpy package - https://gitlab.com/adityadivekar/sign-message
The original dkimpy package hosted on launchpad provides the functions for
DKIM signing and verifying of a message. This module extends the
functionality by adding support for the ARC set of headers.

Now the next step would be to change the namespace and make it more general
to accommodate the ARC capability. That is the module now provides the
capability for ARC protocol implementation but still retains its 'DKIM'
namespace.
Since this would involve quite a few changes, and could easily break the
setup, I would like your thoughts on this if there are any good practices
to follow here.
The way I can think of doing this is to grep all files and change the names
and then clear all the cache files.

Thanks
Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Regarding System set up

[Aditya Divekar]

Hi!

It seems you ran into the error due to the incompatible falcon version.
You should have falcon==0.3 installed for the setup to work for older
Mailman versions.

However, this error doesn't occur with the latest Mailman version 3.1.0
So you should upgrade your Mailman version and the issue would be solved.
You can pull the Mailman master branch from Gitlab and then re-run the setup.

Aditya.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] mailman

[Aditya Divekar]

I'm sorry for the misdirect. I'm myself a beginner here!

This issue on gitlab is probably the same as the one you are facing. The
fix recommended is the installation of falcon 0.3.0.1 or 0.3
https://gitlab.com/mailman/postorius/issues/122#note_4509503

If this doesn't help we'll wait till someone more knowledgeable than me
comes along :)

Thanks!

Aditya Divekar
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] mailman

[Aditya Divekar]

Hi!

Try removing the /mailman3 part from the URL. i.e. try accessing the URL -
http://192.168.122.147:8000/ <http://192.168.122.147:8000/mailman3/>
I'd gotten a similar error. Hope it helps.

Aditya Divekar
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Correcting the display name in the recipient address

[Aditya Divekar]

> A perhaps more convincing example would be an extension to anonymous
>> lists with personal reply-to.  This could be accomplished by creating
>> a mailbox at the list host, which would be associated with the user so
>> she can set options for it (eg, which address to forward to), but
>> should not have a display name associated with it, and certainly not
>> the user-level display name (which in organizational contexts --
>> including Python! -- is likely to be a real name).
>>
>
My knowledge of anonymous mailing lists is a bit less. But from what I
understand, the address for the mailbox would not have a display name for
anonymity purposes whereas the user can have a display name, importing
which may cause anonymity concerns.
The case we handled was regarding the welcome message. The display name
property of the address was not actually changed. So if the user has not
supplied a display name for the address, the display name will remain empty
in that case. Only the welcome message will temporarily use the user
display name. Would that still be a problem in this case?

I might be wrong here if my interpretation of the example is wrong!

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Correcting the display name in the recipient address

[Aditya Divekar]

Stephen J. Turnbull  wrote:

> BTW I made a few more comments on your proposal (after you
> finalized).  I'm told you are allowed to revise the final copy until
> the deadline, it's just that we can't see it until then.  I think it's
> basically in good shape.
>
>
Yes, I checked the comments. Thanks a lot for the input! I had gotten a bit
worked-up about the possible server issue and so uploaded a copy earlier.
I revised the proposal according to your recent comments and re-uploaded it
now. Also, I've replied to a couple of the comments. Specifically, I was
not able to understand your comment in the testing section, which can be
discussed if time permits :)

If by "implicit" you mean "automatic, default", yes, but IMHO there
> should be a distinction between "None" and "InheritFromUser".  That
> will make it easier to "fix" in the future if it turns out that there
> are more than a very few people who feel more strongly than me and
> Mark.
>

> A perhaps more convincing example would be an extension to anonymous
> lists with personal reply-to.  This could be accomplished by creating
> a mailbox at the list host, which would be associated with the user so
> she can set options for it (eg, which address to forward to), but
> should not have a display name associated with it, and certainly not
> the user-level display name (which in organizational contexts --
> including Python! -- is likely to be a real name).
>
>
Actually, the code has already been merged for the above mentioned
automatic fallback on the user display name. So I'm not really sure about
the next step in this issue, and think that Barry would be better qualified
to give an opinion here than me.


> Also, in documentation please always write "the user's display name".
> "User name" has a strong connotation of "account identifier", and so
> could be quite confusing (especially since we don't have a user name
> separate from email addresses!)
>
>
Yes, it was a slip while typing the email. The docs are okay :)

Thanks.

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Correcting the display name in the recipient address

[Aditya Divekar]

Hi!

I certainly agree as far as mining for display names in linked, but not
> subscribed, addresses.  However, I do want to keep the intended semantic
> that
> if the *subscribed* address has no display name, we fall back to the linked
> user record.  The idea is that a user can set their display name once, in
> their user record, and won't need to set it every time they link (and
> validate) a new address to their profile.
>
> Aditya, do you think you have enough information to finish up mr !104?
>
>
So according to what I've gathered from the conversation, the fall back on
the user name can be made implicit. The user name is to be always used in
case of missing display name for the subscribed address.
The feature for suggesting the display names from the other records i.e.
linked via the user, but not subscribed, could, as suggested, be raised in
postorius for the UI part.

@Barry The mr code would first check if a display name is provided while
subscription i.e. `member.address.display_name`.
If not, it will check if the linked user has a display name and use that if
it exists i.e. `member.user.display_name`. And as a final case when both
these fail, it will use an empty string.
I'll go ahead and make these changes.

Thanks!

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Correcting the display name in the recipient address

[Aditya Divekar]

This is regarding the solution for issue #194
 in mailman-core
The discussion for it can be found in the related merge request here

As suggested by Barry, the discussion has been shifted here.

Whenever an address is subscribed to a mailing list, we need to assign a
recipient name to the outgoing welcome mail. eg. To: Anne 
'Anne' being the recipient name.
Now if the address is subscribed without a display name, we should look for
other options for the display name.

Suppose you have a user with two linked addresses, one of which is the
preferred address. Only the preferred address has a display name, and the
user does not. Now in the scenario where the other linked address(not the
preferred one) is subscribed to a mailing list without providing a display
name, it would be natural to check if the user has a display name, and use
that if it exists. But the user too does not have a display name in this
case. In such a case should we perform a further check and try to use the
preferred address's display name if it exists or simply use an empty string
instead?

In the opinion I gave, in case we are not supplied with a display name, we
should check the user display name followed by the preferred address
display name, and use them in the specified order respectively. If none
exist, empty string would be used.

In a more general scenario, as pointed out by Barry to question this
method, suppose a user has multiple linked addresses, only one of which has
a display name, and its neither the preferred address, nor the subscribed
address. Should the display name search algorithm find that one too, since
it still is a linked address for the user and hence contains valid user
information?

Thanks.

Aditya.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

Hi Steve,

After reading the recent mails it seems that I should have submitted the
draft on the gsoc website.
I shall do the same immediately :)


Aditya



On Wed, Mar 16, 2016 at 10:04 PM, Aditya Divekar 
wrote:

> Hi Steve,
>
> I went ahead and drafted a rough proposal as you advised.
> I've included the schedule with the deadlines in the proposal, and
> mentioned the dates with the purpose of giving an approximate idea of how
> the project will be executed.
> The link to the proposal is attached below, and it would be great if you
> could put in your comments in the proposal regarding any opinion that you
> have of it.
>
> Proposal
> <https://docs.google.com/document/d/1dxf_8bdhHkIGJPNsLQy4qdBWxgHQP8ARvZUupR2zN6s/edit?usp=sharing>
>
>
> I haven't submitted this docs link in the summer of code page yet, as I
> would like to get your review first regarding it.
>
> Also,
>
>>
>> I'm going to be out of circulation for another 36 hours or so.  I did
>> want to comment on your new milestones but have other things to take
>> care of so it will have to wait.
>>
>>
> Yes, we could discuss the milestones and depending on the discussion I can
> modify the proposal before its final submission.
>
>
>
>> You haven't mentioned other applications.  Google rules allow up to 5,
>> to different projects with one organization or to multiple
>> organzations.  I recommend against submitting more than 3 as a waste
>> of your energy and probably will detract from your evaluation on all
>> of them.  But nobody's feelings will be hurt if you do submit to other
>> organizations.  I would appreciate knowing as early as possible if you
>> have other prospects (in or out of GSoC).  This will not affect your
>> evaluation in Mailman.
>>
>
> I will be submitting only one proposal, that is in Mailman. Actually, I
> haven't yet been able to find any other projects suitably interesting for
> submitting a proposal.
> So Mailman will remain as the top priority submission for me!
> Also, I will be having my college summer vacations from mid May to July
> end, and have no prior commitments for this period, and in August since
> college will have just reopened I will have enough time to work on the
> project.
>
> I did not know if the proposal is to be CC'ed to the mailing list at this
> stage, and since the previous mail did not CC the mailing list, I have
> followed suit. Hope its not a problem :)
>
> Thanks!
>
> Aditya.
>
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

Regarding my previous mail,
I have implicitly assumed that the testing would involve the use of
examples which do not have a pre-existing valid ARC chain in them i.e. the
test examples would not be containing mails which have multiple ARC set of
headers, and that the ARC chain would be first initiated by Mailman. Such
examples cannot be used for testing before all the milestones are completed
due to the arc verification test being implemented at the end.
We can, however implement such examples in the testing process once the
entire code is completed, since then the arc verification will be in place.
Also, this can be added as a milestone after all the previously mentioned
milestones are complete or can be merged with milestone 8.

Thanks.

Aditya.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

Hi Steve,
As discussed in our earlier conversation, I went over the milestones again
and made some changes.
We would need to use the approach of creating our own examples in the
milestone for the AS code (milestone 6, for testing) too, since the AS
signs over the previous generated set of headers. This will however not be
a problem since the AMS and AAR code should be already completed by that
point thus enabling the creation of test example.
And as said before, the second time this approach would be used in
milestone 7. The reordering of the arc verification after the completion of
the AMS and the AS should not be a problem since the AMS and AS code do not
refer explicitly to the arc verification field in the AAR. Both of them use
the AAR in their signing[1] as a whole field and are not concerned with its
individual fields, and hence the addition of arc at a later stage should
not create any problem. Also the examples used for testing by us in
milestone 5 and 6 would not involve the `arc = ` field since the example
mail will be consisting of the first instance of the ARC headers i.e. `i =
1`, and hence the arc verification test would not be performed (i.e. no
`arc = ` field in the AAR), even after the code for it is inserted in its
place, thus keeping the tests in milestones 5 and 6 valid.

Milestones -

1. ARC Authentication Result - spf verification code completed. tests
passed. merge request created.

2. ARC Authentication Result - dkim verification code completed. tests
passed. merge request created.

3. ARC Authentication Result - dmarc verification code completed. tests
passed. merge request created.

4. ARC Authentication Result - generate AAR header from the previous
milestones code. tests passed. merge request created.

5. ARC Message Signature code completed. tests passed. merge request
created.

6. ARC Seal code completed. tests passed. merge request created.

7. ARC Authentication Result - arc verification code completed. tests
passed. (The workflow is modified to make the mail pass through this
function after milestone 3 [2] ). merge request created.

8. The mail is prepended with the complete set of ARC headers. Tests for
checking the entire ARC set of headers passed. merge request created
(tests).

Comments on testing -

1. For spf testing  - The spf library itself provides test examples, and
mails can be extracted (i.e. the part before gmail adds its own
authentication results) from gmail using the `Show Original` feature. These
mails can be used for testing purposes.

2. For dkim testing - The dkim library provides testing examples, and
similar to above, mails may be used from gmail.

3. For dmarc testing - The examples for this can be gathered from our gmail
account. Any mail sent from yahoo's domain to a gmail account
will have dmarc verification performed. A mail extracted so can be used for
testing purposes by us.

4. Testing this would require testing of the format of the generated AAR
against the official RFC7601 format for authentication results (since the
values of the individual fields have already been tested above) i.e.
keeping track of the delimiters, white spaces, and the correct prepending.
The AAR header generated for an example mail from the above code can be
used here, and tested against the format outlined in the ARC draft.

5. ARC Message Signature testing - We can generate the signature for a
given example mail from the written code, and generate a signature using
the same set of headers from the dkim package (keeping track of the
implicit and explicit headers). The crucial part would be matching the
signatures, i.e. the "b" tag, and matching the body hash, i.e. the "bh"
tag. The "t" tag would differ since it depends on the exact time, and the
"i","a","s","d", "h" tags can be checked manually.

6. ARC Seal testing - For this we could use the previously defined AAR and
AMS, and the current AS code to generate a mail with the full ARC set of
headers (the AS would sign over the AAR and the AMS generated in the
previous milestones). An example mail could be then passed through the
above code, and then can be used for testing purposes (Here we would need
the approach of generating our example from the code as you mentioned).
Here too, the "i", "a" , "s" , "d" , "cv " tags can be checked manually and
the "t" tag would differ. Hence the crucial tag would be the "b" tag.

7. The code for the arc verification test is completed. We can now perform
the arc verification test here. A fully cooked mail with the ARC headers
can be prepared from the above code for the `pass` test . For a `fail`
test, we can make some manual changes in the AS to invalidate it, and then
perform arc verification on it.

8. Here the testing for the complete ARC set of headers would need to be
performed. An example mail can be passed through the above entire code, and
manually asserted in the test. (Since the individual fields would have
already been tested, the main aim here would be testing that the ARC

Re: [Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

Hi Steve,


> The problem is how do you get such a message?  The easiest way is if
> you have implemented the creation functions for the ARC-* fields.
>
> Then you can check that you can roundtrip your own fields (create and
> verify).  Of course you will eventually need to test against other
> implementations (currently Google and AOL have implementations that I
> know of), but for now it makes sense to "eat your own dogfood"
>

Yes, in that case the arc verification would need to be postponed to after
the entire code for the header generation has been written. I'll need to
rethink the milestones a bit, since this testing part would have to be
added at the end, and things will change quite a bit.
I'll work this out properly and reach back to you.

On a side note, would posting on the arc-discuss list for help in the above
problem of testing be acceptable?

Thanks

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

> Stephen Turnbull writes:
> This is all already in the message or msg_data objects, except for the
> value of "i" itself.  So I'm not sure what you're saying.  My point is
> merely that any header field mentioned in any of the relevant RFCs
> that can be validated must be validated to conform to ARC.
>
> By extraction I meant that the message could be parsed to get back a tuple
of "headers, body" where the headers would be returned as lists containing
the name and value. And then all the required headers could be used for the
authentication purposes.
But since we will be receiving a processed mail from Mailman (msg_data), as
mentioned, the parsing won't be required.


> I'm not sure you will need so many separate modules; each verification
> depending on existing standard can probably be kept to a single
> function, unless you can't get all relevant information from the
> upstream modules.  But it doesn't hurt to separate them now, and think
> about combining them later, if that's convenient for you.
>
>
Yes, I meant to refer to separate functions here actually.
By modules I meant that the code for each would be independent, and can be
implemented as processes that the mail through.
Creating separate functions would help in debugging processes. Sorry for
the slip.


>  > 4. ARC Authentication Result - arc verification code completed. tests
>  > passed. merge request created.
>
> How do you propose to create tests for #4?
>
>
Here, we could consider a few scenarios-
1. A message is passed to the function which does not have any previous ARC
headers. In this case the verification is not performed.
2. A message containing a valid ARC chain is passed to the function. In
this case the verification is performed and the most recent ARC Seal is
tested for authenticity, which can be checked in the test.

This would get difficult if we are dealing with cases where the previous
ARC chain is longer than 1 or 2, since it would involve extracting all the
ARC headers in the message ( tricky, since all the headers have the same
"tag", i.e. - all the headers would be named the same - "ARC Message
Signature:", "ARC Seal:", and can be differentiated only on the basis of
the "i" value ), since the most recent ARC Seal will sign over all these,
and hence we need them for the verification process.

Here, a case could occur (if possible) where the message contains a broken
ARC chain, i.e. the last MTA did not add its ARC set of headers. This would
happen if it is assumed that not all MTAs are enforcing ARC.
It is not clear from the draft to me if in this case the arc verification
would have to be performed, since the draft assumes that all the
intermediaries participate in the ARC system. According to me since the
chain is broken, it would not be required to verify the ARC headers.
I would like to know how you think this case can be handled.

Thanks

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

Hi Steve!


> This should be trivial to do with the Python email package, too.  I
> don't really see that a separate module would be useful, since we'll
> want to extract a fixed set of headers (ARC- and DKIM-specified).  Of
> course it should be factored into a separate function (or perhaps a
> generic "extract_fields" function and a couple of derivatives with the
> DKIM list (just DKIM-Signature?) and the ARC list (ARC-Seal,
> ARC-Authentication-Results, and ARC-Message-Signature).
>
>
Depending on the message, if it has previous Authentication Results added
as a header, that can be extracted too.  The entire message can be parsed,
and then all the possible headers involved in the authentication process,
ie. DKIM signature, Authentication results, ARC headers,  can be
 extracted. If not found, a suitable flag can be set for them. ie. example
- if no previous arc headers were found, a flag can be set. This can later
be used in deciding the flow of the mail such as the "i" tag value, whether
we need to perform ARC authentication for the previous ARC headers, and
other fields that depend on the occurrences of any previous ARC set.

Another factoring issue: should you "import dkimpy" and call

> dkimpy.foo, or should you "from dkimpy import foo, bar, baz"?
> (Doesn't need to be settled for a while, and you can even try both at
> a small cost in time and effort.)
>
> From what I've read, I should use the `from ... import ...` when the no of
methods required from the package is less, around 3-4, the entire package
is not required and when naming conflicts are to be minded.
But yes, I can settle it later during the coding part :)

 >Isn't this inaccurate?  This "i" must *match* the Seal "i", no?

Yes, I meant the same thing. The "i" tag for all the three AAR, AMS and the
AS will be the same , and will be one higher than the previous instance of
the "i" tag. If no previous instance is there, it will be 1.

 > a,t,s,d: These can again be obtained from the dkimpy package.
>  >
>  > bh: The body hash. This can be obtained from the package. Here, we set
> the
>  > canonicalization to 'relaxed' and get the body hash.
>
> "The package" == dkimpy, right?
>
> Yes.


>  > 3. ARC Authentication Results.
>  > The "i" tag simply takes on the value same as the above "i" tags.
>  > Now from our previous conversation, as you suggested, the
>  > authenticity of the previous MTA who sent us the mail is not sure
>  > to be trusted. So in the case where we don't trust the previous
>  > MTA, we will have to perform our own DMARC, SPF, DKIM testing of
>  > the recieved mail. If previous ARC chain exists i.e. cv=V, then we
>  > perform the ARC test too.
>
> I'm not sure you understood me.  We *always* verify the preceding
> MTA's claims, even if we trust them, because of spoofing and
> man-in-the-middle attacks on the Internet itself.  (More precisely,
> verification is required by the I-D, and the rationale is spoofing and
> MITM.)  The MTA we "may or may not trust" is our *own* MTA.
>
>
Yes. So for every mail we receive, we always perform the authentication
tests for spf,dkim, dmarc (and arc if present).


> No.  The mail cannot be proved authentic by ARC, but that may be due
> to changes to the message at intervening hops.  There may be more
> sophisticated tests (eg, a PGP signature on a MIME body) that can
> prove it authentic.
>
>
Okay.! So we need to only include the ARC Headers and forward the message
to the subscribers, and
leave it upto their MTAs to do the needful.

 > Now coming to the testing part. There can be a number of tests like
>  > verifying the generated ARC signature, changing the body of the
>  > message, failing when the implicitly signed AMS headers are changed
>  > and other such tests.
>
> This is a little vague, but testing is hard.  You'll learn it as you
> go along.
>

I will come up with at least a few concrete tests that need to be performed
for each of the modules given below (in the milestones), and run them by
you once before the proposal.

>
> I count about 9 well-defined tasks in your message that could be made
> into good milestones.  You should try to come up with your own list,
> but if you're really not happy with your list, send it to me and ask
> for help and I'll give you my ideas.
>
>
I have come up with the following probable milestones for the project. -
(The project has been divided into milestones on the basis of the separate
modules that will be created. Each module is a milestone).
1. ARC Authentication Result - spf verification code completed. tests
passed. merge request created.
2. ARC Authentication Result - dkim verification code completed. tests
passed. merge request created.
3. ARC Authentication Result - dmarc verification code completed. tests
passed. merge request created.
4. ARC Authentication Result - arc verification code completed. tests
passed. merge request created.
5. ARC Authentication Result - generate AAR from the previous milestones
code. tests passed. merge r

Re: [Mailman-Developers] Mailman-bundler installation

[Aditya Divekar]

Hi,

Thanks Simon. I followed the docs and now Mailman 3.0 is up and running :)

And I'll keep that in mind henceforth @Daman Singh. Thanks.

Aditya.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Mailman-bundler installation

[Aditya Divekar]

Hi,
I recently reinstalled my OS, ubuntu 14.04 , and was trying to get the
development server for mailman running. (I had mailman running in my
previous OS setup, but I had to reinstall my OS due to some python issues)
For this I was following the mailman-bundler documentation. After
activating the python2.7 virtual environment, when I give the command to
start mailman, I get an error.
(venv) aditya@aditya:~/mailman-server/mailman-bundler$ sudo
./bin/mailman-web-django-admin runserver &
[1] 3299
(venv) aditya@aditya:~/mailman-server/mailman-bundler$ INFO Generating
grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
INFO Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
INFO Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
INFO Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Performing system checks...

System check identified no issues (0 silenced).
March 07, 2016 - 20:43:38
Django version 1.8.11, using settings 'mailman_web.development'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
/home/aditya/mailman-server/mailman-bundler/mailman_web/urls.py:7:
RemovedInDjango19Warning: Default value of 'RedirectView.permanent' will
change from True to False in Django 1.9. Set an explicit value to silence
this warning.
  url(r'^$',
RedirectView.as_view(url=reverse_lazy('hyperkitty.views.index.index'))),

WARNING /home/aditya/mailman-server/mailman-bundler/mailman_web/urls.py:7:
RemovedInDjango19Warning: Default value of 'RedirectView.permanent' will
change from True to False in Django 1.9. Set an explicit value to silence
this warning.
  url(r'^$',
RedirectView.as_view(url=reverse_lazy('hyperkitty.views.index.index'))),

Traceback (most recent call last):
  File "/usr/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
  File
"/home/aditya/mailman-server/mailman-bundler/eggs/Django-1.8.11-py2.7.egg/django/contrib/staticfiles/handlers.py",
line 63, in __call__
return self.application(environ, start_response)
  File
"/home/aditya/mailman-server/mailman-bundler/eggs/Django-1.8.11-py2.7.egg/django/core/handlers/wsgi.py",
line 170, in __call__
self.load_middleware()
  File
"/home/aditya/mailman-server/mailman-bundler/eggs/Django-1.8.11-py2.7.egg/django/core/handlers/base.py",
line 50, in load_middleware
mw_class = import_string(middleware_path)
  File
"/home/aditya/mailman-server/mailman-bundler/eggs/Django-1.8.11-py2.7.egg/django/utils/module_loading.py",
line 26, in import_string
module = import_module(module_path)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named middleware


I tried checking for the solution online, but couldn't find a fix. I
reinstalled django twice to ensure that the installation was thorough, and
that all the libraries were installed.
Has anyone faced this issue?
Note - I am currently running django 1.9.4 in both python3 and python2.7,
and have installed django 1.8.11 in the 'venv' virtual environment.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] CAPTCHA support

[Aditya Divekar]

I was thinking of this as a side project to work on, not related to gsoc.
And I would be interested in working on this in case it's started as a
project :)
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Basic workflow of the ARC implementation

[Aditya Divekar]

Hi Steve!

Now going back to the implementation, the message needs to be signed after
its headers have been cooked, since they are included in some of the
signatures below.
So after the message headers have been cooked, and before the message has
been enqueued by the switchboards onto the various queues, we shall
catch the email, and perform the ARC operations on it as given below. After
this we can reinject the mail into the queues. This can be done by
modifying the process that handles the queuing of the mails.(This can be
done when the integration process into Mailman starts later on)

For now, I will assume that only the message is available to us in its
complete form after the list has cooked its headers.

We need to generate a private and a public key for the signing purposes.
For testing purposes, and while working on the code, I can probably
generate the keys locally using the openssl tool.

As a rough sketch of implementation,

1. ARC Seal

The tags -

NOTE - Now, here we needed to check the given message for any pre-existing
signatures in most of the fields. For this I think a separate module can be
created which can extract the previous ARC headers if they exist from the
message. The code for this can be again used from the dkimpy package.

i: The value for this tag can be determined by performing a check on the
original signature
and seeing if there were previous ARC headers. If yes, we increment the
value of the previous "i" by 1,
and if no, make it 1.
a: The value for this is fixed.

t,s,d: These can be obtained by using pre-existing dkimpy package.

b: Now we can compute the header hash using the dkimpy package again by
using the headers given here
. Here,
we call the dkimpy package and get the signature for the above headers and
then affix it to the "b="header.

cv: Use the same check as "i", if there is already an ARC i.e. i>1, then we
make it as "V", else "N".

The ARC Seal gets computed here.

NOTE-For giving the "s" and "d" (selector and domain tag values), we will
need to produce records for these where the key can be stored so that it is
available for query by the verifiers (I still have to look up this
mechanism).

2. ARC Message Signature

The tags -

i: The value for this tag is determined similar to the "i" tag for the ARC
Seal.

a,t,s,d: These can again be obtained from the dkimpy package.

bh: The body hash. This can be obtained from the package. Here, we set the
canonicalization to 'relaxed' and get the body hash.

h: The "h=" header list is signed with the implicit list (as given in the
draft) and any explicit list that we want in addition.
Now, for mailing lists, the recommended headers are -
List-Id:List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe:Reply-To:
and any other fields added by the list like the Precedence, XTopics or
else.
We will also sign the DKIM signature of the previous mail here if
available(suggested by the draft).
For this signing, we can use a modified version where the
FROZEN_HEADERS(headers that are signed by default) will specify the
implicit headers according to ARC
specs(Another option could be to get all the implicit+explicit headers
signed by the package, extract the h header, and modify it to include only
the explicit headers.)
b: The signature, calculated from the package. The h tag is used as
described above.


3. ARC Authentication Results.
The "i" tag simply takes on the value same as the above "i" tags.
Now from our previous conversation, as you suggested, the authenticity of
the previous MTA who sent us the mail is not sure to be trusted. So in the
case where we don't trust the previous MTA, we will have to perform our own
DMARC, SPF, DKIM testing of the recieved mail. If previous ARC chain exists
i.e. cv=V, then we perform the ARC test too.
Now for performing the tests -
In one of the earlier mails, we discussed the use of the "authres" package
for generating the authentication results header. The package conforms to
the RFC7001 format, and now the format used is RFC7601. But according to
the changes that I verified, we can use the package without any changes.
(The changes were mostly related to extra specifications that are optional.
Can be skipped for our purposes)
So the "authres" package can be called here for generating the AAR.

If we need to perform the ARC test, then the module for that will have to
be implemented manually. Though most of the code from the package for DKIM
verification can be used.
This is also the point where we detect if the mail is spam or not. If the
arc test fails, then there is something fishy here. DKIM, DMARC, SPF may
fail, but the failure of this test means the mail is not authentic. At this
point the mail should probably be discarded (or any other measures that
need to be taken).

Now coming to the testing part. There can be a number of tests like
verifying the generated ARC signature, chan

Re: [Mailman-Developers] CAPTCHA support

[Aditya Divekar]

Yes, I was of the opinion that it could be an optional feature.

Making it compulsory would involve critical problems as mentioned above by
Christian, and might sometimes be disagreeable to list users who think its
an unnecessary overhead of time.

I was mostly basing this idea on the lines that it is easy to put in a spam
subscription request to the mailing lists now.
Implementing captcha could nip most of these attempts in the bud itself,
and save mailman the extra efforts of sending the confirmation link to the
subscriber.

Thanks.

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] CAPTCHA support

[Aditya Divekar]

Hi!
I was looking around the mailman code, and could not find the functionality
for captcha in the mailing lists subscription pages.
I think it could be a good feature to implement in the upcoming versions,
and would like to know if its a good idea?
I also came across the thread -
http://permalink.gmane.org/gmane.mail.mailman.user/74167
and read about the previous discussion on it by some members.
Since captcha back then was easier to break, it might not have been a
profitable feature, according to the thread.
But with the new recaptcha, I would like to know if the stand is the same.

Thanks

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] ARC module implementation [was: GSOC 2016]

[Aditya Divekar]

Hi!
Sorry for the late reply. I was caught up in my mid term exams.

Stephen J. Turnbull wrote:


> In Mailman 2, there are a number of options.  The one I recommend is
>
adding aol.com and yahoo.com to the subscription and poster ban
> lists.<0.7 wink/>  This has the advantage (?) of also being usable in
> Mailman 3.  A second is to configure the list without transforming the
> message at all (no Subject tags, header, or footer), thus preserving
> any valid DKIM signature.  This also works in Mailman 3.  These are
> standard configuration options which you might use anyway for
> completely different reasons.  However, they sort undermine the
> argument for using Mailman in the first place for many sites.
>
> Yes, I think these are more of workarounds than solutions, if I'm
qualified enough to say so!


> Finally, the most sophisticated alternative is to parse the address
> out of From, and do the DMARC DNS dance to determine if the sending
> domain has a p=reject or p=quarantine policy.  If so, use either the
> From-munging strategy or the MIME-wrapping strategy.  These still have
> the disadvantages described above, but they only apply to posts From
> domains abusing DMARC.
>
> Since the receiver is going to get the mail through the mailing list
anyway, personally I think that the From munging strategy is not a bad
idea. The only case where I can imagine this will create a problem is if I
am receiving the mail as a list posting, and that suppose I want to get all
mails that  sends to the list in a different folder, then
I might face problems since the from header is changed, not allowing the
from filter. I am not sure if that problem will be encountered that often
in context of mailing lists since we usually read all mails from  the list,
(since that is approximately the point behind mailing lists) unless we skip
some by their subject.  (I personally haven't used such a feature, since I
tend to keep all non-me-referencing general mail from the list in the same
folder.) I might be wrong here.

P.S. Also, I have noticed that there is a feature defined in the handlers
to cleanse all the dkim headers from a mail sent to the list before it is
forwarded to the subscribers. We could simply use this to extract the
previous dkim signature, and generate the ARC authentication results since
its a copy of the previous results.


Aditya.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Accessing the members of a list

[Aditya Divekar]

Thanks again Harshit!

And, @Simon, thanks for the input. Also I apologize for dragging the issue
#88  for so long. I am
currently facing problems with my system python3 due to some bad changes.
As a result I'm not able to run mailman3.0 now. I planned on running it and
recording a new request in the vcrpy cassette for the fix, but currently
I'm not able to. I will look into getting the python3 fixed at the
earliest, and close the issue for once.
Thanks for the patience!
Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Accessing the members of a list

[Aditya Divekar]

Thanks for the help Harshit!
But I realized that I framed my question improperly. I used the above
command before too, but it raises an error -

CannotOverwriteExistingCassetteException: No match for the request
(http://localhost:9001/3.0/members/find?subscriber=%2Aexample.com%2A&list_id=foo.example.com&role=member>)
was found. Can't overwrite existing cassette
('/home/hp/postorius_aditya/src/postorius/tests/fixtures/vcr_cassettes/ListMembersTest.test_search_members_1.yaml')
in your current record mode ('once').

Is this error because the request is not present in test cases or is it
some other error?
Also, if it is a test case problem, can anyone suggest me how to change the
test case (or where I could read about how to go about it) to make the
build pass?

Thanks,
Aditya.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Accessing the members of a list

[Aditya Divekar]

Hi,

This is in regard to postorius. Is there a way by which I can access all
the members of a list using its 'list_id' in the list.py file with/without
creating a new client?

The approach I tried was creating a client in the list.py file for this
purpose and calling on the 'members()' function from the Client class
defined in client.py. The doubt I had here is that the members() function
uses the self parameter to decide which list is being referred to. For
this, I think I need to pass the exact url to the list I wish to search for
as a parameter while creating the client and then call on the members()
function on it. Here, how do I figure out the url for this purpose?

Thanks!

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] ARC module implementation [was: GSOC 2016]

[Aditya Divekar]

Hi Stephen,

Stephen J. Turnbull wrote:

>
> I don't think it's a great idea to change the dkimpy library if you
> can avoid it.  The design I had in mind would add a file for
> performing ARC operations such as generating fields, and for dkimpy to
> simply provide the functions for generating the signatures.
>
> I have taken your suggestion, and started working on a separate branch to
generate the ams from the original dkimpy package.
I've created a directory where I can add all the files to generate (or
edit) the signature from the dkim library.

This probably requires some refactoring of dkimpy.  It might also
> require generalizing the canonicalization functions to handle the
> ARC-specific field sets if there are any hard-coded headers in the
> DKIM code.  You might also look at the alternatives to see if they are
> better factored for this purpose.


Yes, some minor changes will be required I believe, like changing the
default frozen headers to be signed or the tags to be added in the sign.
Still, some of them can be circumvented by editing the generated signature
in a separate file.
And yes, I'll go through the other packages you suggested before to see if
I am able to read them better than dkimpy.

>
> I haven't done cross-host work before, but you will surely need a
> gitlab account eventually to file the merge request.
>
> I will try to take a look at the code over the weekend.  I did want to
> make the comments on the factoring of the code ASAP, before you do too
> much work.
>
> I have actually just started working on the repo, and hence there are only
a couple of files that I have committed now.
And yes, I've just pushed the files to github as of now, so that I am able
to get your insight in the proceedings. I'll start working in gitlab soon
once I'm able to come up with a sketch of things.

Thanks!

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] ARC module implementation [was: GSOC 2016]

[Aditya Divekar]

Hi Steve,
I have read through the dkimpy library functions and it seems that the code
needs to be modified to generate the ARC headers, and I have started making
some changes. I'm making these changes in view of generating the AMS for
now using the sign(...) method of the library that is by default used for
generating the dkim signature. Since the AAR is simply a copy, it can be
picked up using a separate method and appended at the end of the AMS later.
I have hosted the code at github here.
, and will keep posting the
changes there as I proceed. Any input from your side would be great :)
Thanks!

Aditya
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSOC 2016

[Aditya Divekar]

Hi Stephen!
Thanks a lot for your previous mail. It was pretty much everything that I
required to start.

I had a doubt regarding the signing in ARC-Seal that I hope you could clear.

According to the draft, here
<https://tools.ietf.org/html/draft-andersen-arc-00#section-5.2>, the
ARC-Seal header is constructed from the referenced message signatures given
by the "k" tag.
I didn't understand this part clearly. Do we need to specify the headers
(i.e. the ARC-Message signature and ARC-authentication results) that we
want to seal in the "k" tag just how we would use the "h" tag in the DKIM
signature?
(The usage of the "k" tag isn't clear to me.)

Thanks!

On Wed, Feb 10, 2016 at 3:19 PM, Stephen J. Turnbull 
wrote:

> Hi, Aditya!
>
> I've had a chance to read your mail and review the I-D (Internet
> Draft) and relevant RFCs a bit, and can now make a few comments.
>
> First, your understanding is a little bit shallow.  You should get
> yourself a "canvas" and draw a detailed flowchart of what's going on
> here.  I don't mean to be harsh.  There's an important (especially to
> you! ;-) principle here: you're volunteering for *too* *much* work.
> Another point is that you're not giving the I-D authors enough credit
> for caring about you and the end users.  Homework: how do these
> principles apply in the comments below? :-)
>
> Aditya Divekar writes:
>
>  > When we use mailman, the mailing list service adds an extra phrase in
>  > the subject - [Mailman-Developers] and an extra footer in the mail
>  > giving links about the FAQ, archives and the security policy. This
>  > alters the original subject and the body of the mail that the sender
>  > sent in the first place. According to my knowledge, this is what might
>  > cause the mail to be rejected by yahoo, aol, or other p=reject policy
>  > domains.
>
> Close enough for our purposes.
>
>   - If you get interested in doing more mail authentication stuff
> you'll need the more accurate story.
>
>  > Thus implementing ARC would involve including the ARC
>  > authentication result header, the signature and the seal in every
>  > mail that Mailman receives before it forwards it in the mailing
>  > list.
>
> Yes.
>
>  > This would probably involve using the pydkim, gs.dmarc and pyspf
>  > libraries for verification before we generate the ARC
>  > authentication results.
>
> Here's where you start doing too much work.
>
>   - You can (for GSoC) assume that the original authentication results
> should already be available.
>
> - For the common MTAs used with Mailman, the
>   Authentication-Results (A-R) header should be available, it
>   should have been added by *your* MTA (otherwise it was added by
>   someone you shouldn't trust!), and you can detect that.  (This
>   may not be verifiable -- need to check, and it's another case to
>   deal with later.)  The ARC-Authentication-Results is a *copy* of
>   this header (see 5.1.3 in the I-D).  (This is the "DRY"
>   principle at work.)
>
> - The A-R header already contains SPF and DKIM results, and maybe
>   DMARC (that's cheap to check, though).  Thus for verification
>   you don't need to do any work!  (First draft -- pragmatically,
>   some Mailman sites may not implement A-R, and as mentioned you
>   may not be able to trust A-R.  That's why we're doing this
>   anyway -- it's really an MTA function, but some sites won't
>   implement.)
>
> - "For GSoC" is important -- as an extension it's desirable that
>   you handle the case where they're *not* available.  But that
>   comes later.
>
>   - But you do absolutely need to sign things (ARC-Message-Signature,
> ARC-Seal) with new signatures, and that uses DKIM (modified
> somewhat, don't know yet if you'll need to modify package code).
>
> Having implemented that much, you now know a lot about how the
> DKIM module (eg, pydkim) works.
>
>   - Learning DKIM verification is probably easier (the relevant header
> field should tell the module everything about what it needs to
> do!)
>
>   - SPF and DMARC are different modules, but again, you've learned a
> lot.
>
> In general, sequence your plans to learn things that will make later
> work easier.  (This tends to happen naturally as you do the work, but
> you can often buy 10%-25% by planning it in advance.)
>
>  > As a starter I think I should understand how the dkim,dmarc and spf
>  > authentication processes are coded.
>  > coul

Re: [Mailman-Developers] GSOC 2016

[Aditya Divekar]

On Mon, Feb 8, 2016 at 10:50 PM, Stephen J. Turnbull 
wrote:
> Aditya Divekar writes:
>
> > And I had previously mailed you directly, but I think I missed you
> > there. So I mailed it again here.
>
> No, I've just been really busy with work, and a very bad cold,
> sleeping about 2 hours/day more than usual. I'll get back to you
> tomorrow evening my time (it's 2am here now, and in the morning I have
> other promises to keep).
>
> You really should be a little more patient. You don't have any
> deadlines to meet this week (things would be way different if it were
> 48 hours to final proposal submission or 168 hours to a Google
> "payday" deadline). Also, in this community, if you wait a bit and I
> don't answer, somebody else will step up.
>
> And don't forget -- you get paid kilodollars if you get accepted. All
> we get is a T-shirt (and last year mine got lost in the mail ;-).
>
Sorry about the double mails.
And sure, I'll wait :)
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Error in starting mailman

[Aditya Divekar]

@ Abhilash Raj -  Thanks for your swift reply. It worked. I terminated
it wrongly last time I suppose.

@ Mark Sapiro -  Yes, there was a stale lock file not associated with
any process. Removed it. And I'll go through the FAQ. Thanks!

On Mon, Feb 8, 2016 at 10:40 PM, Mark Sapiro  wrote:
> On 02/08/2016 08:42 AM, Aditya Divekar wrote:
>> I got the following error when I was trying to start the mailman service 
>> today -
>>
>> master: error: The master lock could not be acquired.  It appears as
>> though there is a stale master lock
>
>
> The service was not gracefully stopped the last time it was running and
> left the master lock behind. The FAQ at
> <http://wiki.list.org/x/17891756> describes how locks are implemented in
> Mailman 2.1, but Mailman 3 is similar.
>
> There is a var/locks/ directory in your Mailman hierarchy. In it you
> will see two files. in MM 2.1, they would be named master-qrunner and
> master-qrunner.hostname.pid. In MM 3 they will be similar. Assuming
> there is no process with PID = pid running on hostname, you can safely
> remove those files.
>
> --
> Mark Sapiro The highway is for gamblers,
> San Francisco Bay Area, Californiabetter use your sense - B. Dylan
> ___
> Mailman-Developers mailing list
> Mailman-Developers@python.org
> https://mail.python.org/mailman/listinfo/mailman-developers
> Mailman FAQ: http://wiki.list.org/x/AgA3
> Searchable Archives: 
> http://www.mail-archive.com/mailman-developers%40python.org/
> Unsubscribe: 
> https://mail.python.org/mailman/options/mailman-developers/adityadivekar03%40gmail.com
>
> Security Policy: http://wiki.list.org/x/QIA9
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Error in starting mailman

[Aditya Divekar]

I got the following error when I was trying to start the mailman service today -

master: error: The master lock could not be acquired.  It appears as
though there is a stale master lock

It first printed a long script regarding the Master subprocess watcher
and then threw the above error.

It was working completely fine just a day back. Any help would be appreciated.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSOC 2016

[Aditya Divekar]

And I had previously mailed you directly, but I think I missed you
there. So I mailed it again here.
Sorry if I caused any inconvenience.

On Mon, Feb 8, 2016 at 2:26 PM, Aditya Divekar
 wrote:
> Hello Stephen,
> I had earlier contacted you on the developer
> mailing list regarding the gsoc project.
> I have started reading about ARC as you suggested and have thought
> about a few things.
> When we use mailman, the mailing list service adds an extra phrase in
> the subject - [Mailman-Developers] and an extra footer in the mail
> giving links about the FAQ, archives and the security policy. This
> alters the original subject and the body of the mail that the sender
> sent in the first place. According to my knowledge, this is what might
> cause the mail to be rejected by yahoo, aol, or other p=reject policy
> domains.
> Thus implementing ARC would involve including the ARC authentication
> result header, the signature and the seal in every mail that Mailman
> receives before it forwards it in the mailing list. This would
> probably involve using the pydkim, gs.dmarc and pyspf libraries for
> verification before we generate the ARC authentication results.
> As a starter I think I should understand how the dkim,dmarc and spf
> authentication processes are coded.
> could you tell me how to find existing code where I can read and
> understand how the authentication methods are implemented?
> Thanks!
>
>
> Aditya.
>
> On Fri, Feb 5, 2016 at 9:20 AM, Stephen J. Turnbull  
> wrote:
>> Aditya Divekar writes:
>>
>>  > My name is Aditya Divekar. I am a sophomore from IIT Guwahati.
>>
>> Nice to meet you, Aditya!  I'm the main DMARC/IETF wrangler for
>> Mailman, and I would be the main mentor for the ARC project.
>>
>>  > I want to work on the project "Implement module to process ARC
>>  > headers". I have begun reading about RFC a bit.
>>
>> That's a good start.  If you have questions, feel free to ask.  For
>> general questions that are mostly about "how do I hook code into
>> Mailman" or about GSoC, please ask on this list.  Not only will you
>> get better and quicker answers, but the questions and answers will
>> benefit other developers too.  For questions about ARC, you can write
>> me directly or the list, as you feel comfortable.
>>
>> Or once you start to get the feel of things you may try to ask on the
>> ARC list.  However, IEFT lists are probably very different from
>> anything you've participated in before.  High stakes are involved
>> (there are people with millions of dollars invested in servers there)
>> and people can be a little terse.  Not to mention the vocabulary will
>> likely be new to you.
>>
>> The next thing to do would be to join the ARC mailing list and lurk:
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.dmarc.org/mailman/listinfo/arc-discuss
>> or, via email, send a message with subject or body 'help' to
>> arc-discuss-requ...@dmarc.org
>>
>> Right now it's low-traffic.  It's a Mailman list.  I subscribed with
>> the digest, and get maybe one a week.
>>
>>  > Please help me get started in the right direction and if possible
>>  > share some timeline goals.
>>
>> Well, the main timeline goal would be to get done in time for the live
>> test of implementations being held by the DMARC folks -- on Feb 19.
>> So I guess that's not going to happen!
>>
>> It is my belief that a full implementation (with bugs still in it) can
>> easily be done in a summer starting from a reasonable amount of
>> programming skill in Python.  If you're better than average it will
>> probably be integratable into Mailman and ready for participating with
>> other implementations on the Internet at the end of the summer.
>>
>> With that in mind, please read How to SPAM
>> (http://turnbull.sk.tsukuba.ac.jp/Blog/SPAM.txt) and other general
>> information about GSoC proposals at
>> http://wiki.list.org/DEV/Google%20Summer%20of%20Code%202016.
>> Then write something up.  Pretty much anything.  It doesn't have to be
>> complete, it just needs to demonstrate you've thought for a few
>> minutes about what you think you need to do.
>>
>> Yes, this is pretty sketchy.  If you're going to work with me, you
>> need to accept that I'm going expect you to try something plausible
>> before I tell you what I expect.  I'm not a complete curmudgeon about
>> it, but I have found that it is a good way to work for me.
>>
>> Regards, and happy hacking!
>>
>> Steve
>>
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSOC 2016

[Aditya Divekar]

Hello Stephen,
I had earlier contacted you on the developer
mailing list regarding the gsoc project.
I have started reading about ARC as you suggested and have thought
about a few things.
When we use mailman, the mailing list service adds an extra phrase in
the subject - [Mailman-Developers] and an extra footer in the mail
giving links about the FAQ, archives and the security policy. This
alters the original subject and the body of the mail that the sender
sent in the first place. According to my knowledge, this is what might
cause the mail to be rejected by yahoo, aol, or other p=reject policy
domains.
Thus implementing ARC would involve including the ARC authentication
result header, the signature and the seal in every mail that Mailman
receives before it forwards it in the mailing list. This would
probably involve using the pydkim, gs.dmarc and pyspf libraries for
verification before we generate the ARC authentication results.
As a starter I think I should understand how the dkim,dmarc and spf
authentication processes are coded.
could you tell me how to find existing code where I can read and
understand how the authentication methods are implemented?
Thanks!


Aditya.

On Fri, Feb 5, 2016 at 9:20 AM, Stephen J. Turnbull  wrote:
> Aditya Divekar writes:
>
>  > My name is Aditya Divekar. I am a sophomore from IIT Guwahati.
>
> Nice to meet you, Aditya!  I'm the main DMARC/IETF wrangler for
> Mailman, and I would be the main mentor for the ARC project.
>
>  > I want to work on the project "Implement module to process ARC
>  > headers". I have begun reading about RFC a bit.
>
> That's a good start.  If you have questions, feel free to ask.  For
> general questions that are mostly about "how do I hook code into
> Mailman" or about GSoC, please ask on this list.  Not only will you
> get better and quicker answers, but the questions and answers will
> benefit other developers too.  For questions about ARC, you can write
> me directly or the list, as you feel comfortable.
>
> Or once you start to get the feel of things you may try to ask on the
> ARC list.  However, IEFT lists are probably very different from
> anything you've participated in before.  High stakes are involved
> (there are people with millions of dollars invested in servers there)
> and people can be a little terse.  Not to mention the vocabulary will
> likely be new to you.
>
> The next thing to do would be to join the ARC mailing list and lurk:
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.dmarc.org/mailman/listinfo/arc-discuss
> or, via email, send a message with subject or body 'help' to
> arc-discuss-requ...@dmarc.org
>
> Right now it's low-traffic.  It's a Mailman list.  I subscribed with
> the digest, and get maybe one a week.
>
>  > Please help me get started in the right direction and if possible
>  > share some timeline goals.
>
> Well, the main timeline goal would be to get done in time for the live
> test of implementations being held by the DMARC folks -- on Feb 19.
> So I guess that's not going to happen!
>
> It is my belief that a full implementation (with bugs still in it) can
> easily be done in a summer starting from a reasonable amount of
> programming skill in Python.  If you're better than average it will
> probably be integratable into Mailman and ready for participating with
> other implementations on the Internet at the end of the summer.
>
> With that in mind, please read How to SPAM
> (http://turnbull.sk.tsukuba.ac.jp/Blog/SPAM.txt) and other general
> information about GSoC proposals at
> http://wiki.list.org/DEV/Google%20Summer%20of%20Code%202016.
> Then write something up.  Pretty much anything.  It doesn't have to be
> complete, it just needs to demonstrate you've thought for a few
> minutes about what you think you need to do.
>
> Yes, this is pretty sketchy.  If you're going to work with me, you
> need to accept that I'm going expect you to try something plausible
> before I tell you what I expect.  I'm not a complete curmudgeon about
> it, but I have found that it is a good way to work for me.
>
> Regards, and happy hacking!
>
> Steve
>
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] GSOC 2016

[Aditya Divekar]

Hi everyone,
My name is Aditya Divekar. I am a sophomore from IIT Guwahati.
I have recently gotten involved with the mailman project, and gained
familiarity with postorius and mailman core.
I want to work on the project "Implement module to process ARC headers". I
have begun reading about RFC a bit.
Please help me get started in the right direction and if possible share
some timeline goals.
Thanks.

Aditya.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] help in testing

[Aditya Divekar]

I have just started working on the mailman project. I am currently trying
to solve Issue #186 and need help in testing, to check if my recommended
fix is correct. any help would be appreciated.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9